Neural network-based generation and placement of tooth restoration dental appliances

ABSTRACT

Techniques are described for automating the design of dental restoration appliances using neural networks. An example computing device receives transform information associated with a current dental anatomy of a dental restoration patient, provides the transform information associated with the current dental anatomy of the dental restoration patient as input to a neural network trained with transform information indicating placement of a dental appliance component with respect to one or more teeth of corresponding dental anatomies, the dental appliance being used for dental restoration treatment for the one or more teeth, and executes the neural network using the input to produce placement information for the dental appliance component with respect to the current dental anatomy of the dental restoration patient.

TECHNICAL FIELD

The present disclosure relates to dental restoration appliances used forreshaping teeth.

BACKGROUND

Dental practitioners often utilize dental appliances to re-shape orrestore a patient's dental anatomy. The dental appliance is typicallyconstructed from a model of the patient's dental anatomy, augmented to adesired dental anatomy. The model may be a physical model or a digitalmodel. Designing the dental appliance is often a manual, time-consuming,and inexact process. For example, a practitioner typically designs amodel of the dental appliance by trial and error. For instance, thepractitioner may add, remove, reposition, rearrange, and/or resizefeatures until the practitioner is satisfied with the model of thedental appliance.

SUMMARY

The disclosure relates to techniques for automating a design of a dentalrestoration appliance for restoring the dental anatomy of a givenpatient and/or a proposed placement of a dental restoration appliancefor the dental restoration treatment of the patient. Computing systemsconfigured according to aspects of this disclosure implement neuralnetwork-based systems trained with any of a variety of datasets togenerate design aspects (or “geometry”) of a dental restorationappliance for a particular patient and/or the placement characteristics(or “transform”) of the dental restoration appliance for a particularpatient. To generate a custom geometry of a patient-specific dentalrestoration appliance and/or transform information thereof, thecomputing systems of this disclosure implement neural networks trainedwith dental anatomy and/or dental appliance information available fromdigital libraries of predefined appliance geometries and/or pre-madeappliance “ground truths” (e.g. appliance geometries that were createdmanually by a skilled practitioner or created automatically via therules-based system described in WO 2020/240351, filed May 20, 2020, theentire content of which is incorporated herein by reference).

The geometry of a dental appliance is represented by a digitalthree-dimensional (3D) mesh incorporating the features of the generatedgeometry. In various examples, the neural network-based techniques ofthis disclosure may generate a new geometry based on the patient'sdental anatomy and on the dataset(s) of “ground truth” appliancegeometries with which the neural network is trained. In some examples,the neural network-based techniques of this disclosure may generateplacement data for a geometry which is selected from a digital libraryof dental appliance geometries, and then based on the patient's dentalanatomy and based on the dataset(s) of transformation matrices withwhich the neural network is trained, place that library geometryrelative to or more of the patient's teeth.

In some examples, the neural network-based techniques of this disclosuremay generate placement data for a dental appliance which is selectedfrom a digital library of dental appliance geometries. Examples ofplacement data may relate to one or more of the position, theorientation, the scale, or shear mapping of the dental appliance as itis to be placed during dental restoration treatment of the patient. Thecomputing systems of this disclosure may implement a simple neuralnetwork (e.g., with a relatively small number of hidden layers or nohidden layers), a graph convolutional neutral network (GCNN), agenerative adversarial network (GAN), a conditional generativeadversarial network (cGAN), an encoder-decoder based CNN, a U-Net CNN, aGAN with a PatchGAN discriminator, and/or another deep neutral network(DNN) to perform various techniques described herein.

In this way, the computing systems of this disclosure train and executeneural networks to automate the selection/generation of dental appliancecomponent 3D meshes as well as the placement of dental appliance librarycomponent 3D meshes, all of which are later combined and assembled intoa completed dental appliance. The computing systems of this disclosuremay train the neural networks using a variety of data, such as dentalanatomy landmarks, patient-to-mesh mappings, two-dimensional (2D) dentalanatomy images, etc. to define a dental restoration appliance componentand its placement. The geometry and placement information generated bythe neural networks trained with the dataset(s) selected according tothe techniques of this disclosure automatically generate a mesh (a 3Ddigital custom model) which can be used to manufacture thepatient-specific dental restoration appliance, such as by 3D printingthe restoration appliance from the mesh.

In one example, a computing device includes an input interface and aneural network engine. The input interface is configured to receivetransform information associated with a current dental anatomy of adental restoration patient. The neural network engine configured toprovide the transform information associated with the current dentalanatomy of the dental restoration patient as input to a neural networktrained with transform information indicating placement of a dentalappliance component with respect to one or more teeth of correspondingdental anatomies, the dental appliance being used for dental restorationtreatment for the one or more teeth. The neural network engine isfurther configured to execute the neural network using the input toproduce placement information for the dental appliance component withrespect to the current dental anatomy of the dental restoration patient.

In another example, a method includes receiving transform informationassociated with a current dental anatomy of a dental restorationpatient. The method further includes providing the transform informationassociated with the current dental anatomy of the dental restorationpatient as input to a neural network trained with transform informationindicating placement of a dental appliance component with respect to oneor more teeth of corresponding dental anatomies, the dental appliancebeing used for dental restoration treatment for the one or more teeth.The method further includes executing the neural network using the inputto produce placement information for the dental appliance component withrespect to the current dental anatomy of the dental restoration patient.

In another example, an apparatus includes means for receiving transforminformation associated with a current dental anatomy of a dentalrestoration patient, means for providing the transform informationassociated with the current dental anatomy of the dental restorationpatient as input to a neural network trained with transform informationindicating placement of a dental appliance component with respect to oneor more teeth of corresponding dental anatomies, the dental appliancebeing used for dental restoration treatment for the one or more teeth,and means for executing the neural network using the input to produceplacement information for the dental appliance component with respect tothe current dental anatomy of the dental restoration patient.

In another example, a non-transitory computer-readable storage medium isencoded with instructions. The instructions, when executed, cause one ormore processors of a computing system to receive transform informationassociated with a current dental anatomy of a dental restorationpatient, to provide the transform information associated with thecurrent dental anatomy of the dental restoration patient as input to aneural network trained with transform information indicating placementof a dental appliance component with respect to one or more teeth ofcorresponding dental anatomies, the dental appliance being used fordental restoration treatment for the one or more teeth, and to executethe neural network using the input to produce placement information forthe dental appliance component with respect to the current dentalanatomy of the dental restoration patient.

In one example, a computing device includes an input interface and aneural network engine. The input interface is configured to receive oneor more three-dimensional (3D) tooth meshes associated with a currentdental anatomy of a dental restoration patient and a 3D component meshrepresenting a generated geometry for a dental appliance component. Theneural network engine configured to provide the one or more 3D toothmeshes and the 3D component mesh received by the input interface asinputs to a neural network trained with training data comprising groundtruth dental appliance component geometries and 3D tooth meshes ofcorresponding dental restoration cases. The neural network engine isfurther configured to execute the neural network using the providedinputs to produce an updated model of the dental appliance componentwith respect to the current dental anatomy of the dental restorationpatient.

In another example, a method includes receiving, at an input interface,one or more three-dimensional (3D) tooth meshes associated with acurrent dental anatomy of a dental restoration patient and a 3Dcomponent mesh representing a generated geometry for a dental appliancecomponent. The method further includes providing, by a neural networkengine communicatively coupled to the input interface, the one or more3D tooth meshes and the 3D component mesh received by the inputinterface as inputs to a neural network trained with training datacomprising ground truth dental appliance component geometries and 3Dtooth meshes of corresponding dental restoration cases. The methodfurther includes executing, by the neural network engine, the neuralnetwork using the provided inputs to produce an updated model of thedental appliance component with respect to the current dental anatomy ofthe dental restoration patient.

In another example, an apparatus includes means for receiving one ormore three-dimensional (3D) tooth meshes associated with a currentdental anatomy of a dental restoration patient and a 3D component meshrepresenting a generated geometry for a dental appliance component,means for providing the one or more 3D tooth meshes and the 3D componentmesh received by the input interface as inputs to a neural networktrained with training data comprising ground truth dental appliancecomponent geometries and 3D tooth meshes of corresponding dentalrestoration cases, and means for executing the neural network using theprovided inputs to produce an updated model of the dental appliancecomponent with respect to the current dental anatomy of the dentalrestoration patient.

In another example, a non-transitory computer-readable storage medium isencoded with instructions. The instructions, when executed, cause one ormore processors of a computing system to receive one or morethree-dimensional (3D) tooth meshes associated with a current dentalanatomy of a dental restoration patient and a 3D component meshrepresenting a generated geometry for a dental appliance component, toprovide the one or more 3D tooth meshes and the 3D component meshreceived by the input interface as inputs to a neural network trainedwith training data comprising ground truth dental appliance componentgeometries and 3D tooth meshes of corresponding dental restorationcases, and to execute the neural network using the provided inputs toproduce an updated model of the dental appliance component with respectto the current dental anatomy of the dental restoration patient.

In one example, a computing device includes an input interface and aneural network engine. The input interface is configured to receive atwo-dimensional (2D) image of a current dental anatomy of a dentalrestoration patient. The neural network engine configured to provide the2D image of the current dental anatomy of the dental restoration patientas an input to a neural network trained with training data comprising 2Dimages of pre-restoration dental anatomies and corresponding 2D imagesof post-restoration dental anatomies of previously performed dentalrestoration cases. The neural network engine is further configured toexecute the neural network using the input to produce a 2D image of aproposed dental anatomy of the dental restoration patient, the proposeddental anatomy being associated with a post-restoration outcome of adental restoration plan for the dental restoration patient.

In another example, a method includes receiving a two-dimensional (2D)image of a current dental anatomy of a dental restoration patient. Themethod further includes providing the 2D image of the current dentalanatomy of the dental restoration patient as an input to a neuralnetwork trained with training data comprising 2D images ofpre-restoration dental anatomies and corresponding 2D images ofpost-restoration dental anatomies of previously performed dentalrestoration cases. The method further includes executing the neuralnetwork using the input to produce a 2D image of a proposed dentalanatomy of the dental restoration patient, the proposed dental anatomybeing associated with a post-restoration outcome of a dental restorationplan for the dental restoration patient.

In another example, an apparatus includes means for receiving atwo-dimensional (2D) image of a current dental anatomy of a dentalrestoration patient, means for providing the 2D image of the currentdental anatomy of the dental restoration patient as an input to a neuralnetwork trained with training data comprising 2D images ofpre-restoration dental anatomies and corresponding 2D images ofpost-restoration dental anatomies of previously performed dentalrestoration cases, and means for executing the neural network using theinput to produce a 2D image of a proposed dental anatomy of the dentalrestoration patient, the proposed dental anatomy being associated with apost-restoration outcome of a dental restoration plan for the dentalrestoration patient.

In another example, a non-transitory computer-readable storage medium isencoded with instructions. The instructions, when executed, cause one ormore processors of a computing system to receive a two-dimensional (2D)image of a current dental anatomy of a dental restoration patient, toprovide the 2D image of the current dental anatomy of the dentalrestoration patient as an input to a neural network trained withtraining data comprising 2D images of pre-restoration dental anatomiesand corresponding 2D images of post-restoration dental anatomies ofpreviously performed dental restoration cases, and to execute the neuralnetwork using the input to produce a 2D image of a proposed dentalanatomy of the dental restoration patient, the proposed dental anatomybeing associated with a post-restoration outcome of a dental restorationplan for the dental restoration patient.

In one example, a computing device includes an input interface and aneural network engine. The input interface is configured to receive oneor more three-dimensional (3D) tooth meshes associated with a currentdental anatomy of a dental restoration patient. The neural networkengine configured to provide the one or more 3D tooth meshes received bythe input interface as input to a neural network trained with trainingdata comprising ground truth dental appliance component geometries and3D tooth meshes of corresponding dental restoration cases. The neuralnetwork engine is further configured to execute the neural network usingthe provided input to produce a custom geometry for the dental appliancecomponent with respect to the current dental anatomy of the dentalrestoration patient.

In another example, a method includes receiving one or morethree-dimensional (3D) tooth meshes associated with a current dentalanatomy of a dental restoration patient. The method further includesproviding the one or more 3D tooth meshes associated with the currentdental anatomy of the dental restoration patient as input to a neuralnetwork trained with training data comprising ground truth dentalappliance component geometries and 3D tooth meshes of correspondingdental restoration cases. The method further includes executing theneural network using the provided input to produce a custom geometry forthe dental appliance component with respect to the current dentalanatomy of the dental restoration patient.

In another example, an apparatus includes means for receiving one ormore three-dimensional (3D) tooth meshes associated with a currentdental anatomy of a dental restoration patient, means for providing theone or more 3D tooth meshes associated with the current dental anatomyof the dental restoration patient as input to a neural network trainedwith training data comprising ground truth dental appliance componentgeometries and 3D tooth meshes of corresponding dental restorationcases, and means for executing the neural network using the providedinput to produce a custom geometry for the dental appliance componentwith respect to the current dental anatomy of the dental restorationpatient.

In another example, a non-transitory computer-readable storage medium isencoded with instructions. The instructions, when executed, cause one ormore processors of a computing system to receive one or morethree-dimensional (3D) tooth meshes associated with a current dentalanatomy of a dental restoration patient, to provide the one or more 3Dtooth meshes associated with the current dental anatomy of the dentalrestoration patient as input to a neural network trained with trainingdata comprising ground truth dental appliance component geometries and3D tooth meshes of corresponding dental restoration cases, and toexecute the neural network using the provided input to produce a customgeometry for the dental appliance component with respect to the currentdental anatomy of the dental restoration patient.

The techniques and practical applications described herein may providecertain advantages. For example, by automatically determining thegeometry and placement of a 3D mesh to form a component of a dentalappliance or a 3D mesh to form an overall model of a dental appliancefor restorative treatment of a patient, the computing systems of thisdisclosure may improve data precision and conserve resources. Forinstance, by generating a more accurate 3D mesh component for the dentalappliance, the computing systems of this disclosure may improve thefunctionality and efficacy of the dental appliance when used forrestorative treatment.

In instances in which the computing systems utilize a reduced-layerneural network to predict placement information for the dentalappliance, the computing systems may mitigate computational resourceusage, by implementing a neural network with fewer hidden layers. Ininstances in which the computing systems utilize a GAN, GCNN, cGAN, anencoder-decoder based CNN, a U-Net CNN, a GAN with a PatchGANdiscriminator, or other deep neural network, to generate the geometry ofthe 3D mesh, the computing systems provide process improvements byreducing iterations caused by defective or suboptimal dental appliancessupplied to a dental practitioner when performing restorative treatmentfor a patient. In this way, the neural network-based dental applianceconfiguration techniques of this disclosure improve speed, accuracy, andpredictability.

Restoring the patient's dental anatomy more quickly and/or moreaccurately may improve the functionality (e.g., reducing grinding orinterference between teeth), which may improve the patient's quality oflife, for example, by reducing pain caused by suboptimal dentalmorphology, integrity, or functioning. In some examples, restoring thepatient's dental anatomy more accurately may improve the appearance ofthe patient's dental anatomy, which may further improve the patientexperience and/or quality of life. Further, by creating a precise,quick, and predictable process for restoring dental anatomy by way ofthe neural network-formed geometry and/or placement, the computingsystems of this disclosure provide efficiency enhancements for a widerrange of dental practitioners and affordability improvements for a widerrange of patients.

The details of one or more examples are set forth in the accompanyingdrawings and in the description below. Other features, objects, andadvantages will be apparent from the description and drawings, and fromthe claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example system for designingand manufacturing a dental appliance for restoring the dental anatomy ofa patient, in accordance with various aspects of this disclosure.

FIG. 2 is a flowchart illustrating an example process that the system ofFIG. 1 may perform to generate a digital model of a dental appliance byexecuting a neural network trained according to aspects of thisdisclosure.

FIG. 3 is a flow diagram illustrating an example use of a neural networkfor placement of a library component of a dental appliance in accordancewith aspects of this disclosure.

FIG. 4 is a flow diagram illustrating an example of neural network-basedcomponent geometry generation in accordance with aspects of thisdisclosure.

FIG. 5 is a flowchart illustrating a process that a computing device mayimplement to generate component geometries using a GAN, according toaspects of this disclosure.

FIG. 6 is a rendering illustrating an example center clip placementperformed according to the neural network-based placement techniques ofthis disclosure.

FIG. 7 is a rendering illustrating an example of a bonding pad (e.g., ofa lingual bracket) that is customized to the shape of the correspondingtooth.

FIG. 8 is a rendering illustrating an example of a set of componentsthat make up a lingual bracket.

FIG. 9 is a a flow diagram illustrating another example of neuralnetwork-based component geometry generation in accordance with aspectsof this disclosure.

FIG. 10 is a conceptual diagram illustrating the symbiotic trainingprocesses for the generator network and discriminator network of a cGANconfigured to render 2D images of a proposed dental anatomy for apatient according to aspects of this disclosure.

FIG. 11A illustrates the input and output of a cGAN-trained generatornetwork configured to generate a 2D image of a proposed dental anatomyusing a 2D rendering of a current dental anatomy of a patient.

FIG. 11B illustrates a comparison between a current dental anatomyimage, a proposed dental anatomy image of this disclosure, and a groundtruth restoration image.

FIG. 12 illustrates menus that a computing device of FIG. 1 may displayas part of a graphical user interface (GUI) that includes a currentdental anatomy image and/or a proposed dental anatomy image of thisdisclosure.

FIGS. 13A & 13B are conceptual diagrams illustrating example moldparting surfaces, in accordance with various aspects of this disclosure.

FIG. 14 is a conceptual diagram illustrating an example gingival trimsurface, in accordance with various aspects of this disclosure.

FIG. 15 is a conceptual diagram illustrating an example facial ribbon,in accordance with various aspects of this disclosure.

FIG. 16 is a conceptual diagram illustrating an example lingual shelf,in accordance with various aspects of this disclosure.

FIG. 17 is a conceptual diagram illustrating example doors and windows,in accordance with various aspects of this disclosure.

FIG. 18 is a conceptual diagram illustrating example rear snap clamps,in accordance with various aspects of this disclosure.

FIG. 19 is a conceptual diagram illustrating example door hinges, inaccordance with various aspects of this disclosure.

FIGS. 20A & 20B are conceptual diagrams illustrating example door snaps,in accordance with various aspects of this disclosure.

FIG. 21 is a conceptual diagram illustrating an example incisal ridge,in accordance with various aspects of this disclosure.

FIG. 22 is a conceptual diagram illustrating an example center clip, inaccordance with various aspects of this disclosure.

FIG. 23 is a conceptual diagram illustrating example door vents, inaccordance with various aspects of this disclosure.

FIG. 24 is a conceptual diagram illustrating example doors, inaccordance with various aspects of this disclosure.

FIG. 25 is a conceptual diagram illustrating an example diastema matrix,in accordance with various aspects of this disclosure.

FIG. 26 is a conceptual diagram illustrating an example manufacturingcase frame and an example dental appliance, in accordance with variousaspects of this disclosure.

FIG. 27 is a conceptual diagram illustrating an example dental applianceincluding custom labels, in accordance with various aspects of thisdisclosure.

DETAILED DESCRIPTION

FIG. 1 is a block diagram illustrating an example system for designingand manufacturing a dental appliance for restoring the dental anatomy ofa patient, in accordance with various aspects of this disclosure. In theexample of FIG. 1 , system 100 includes clinic 104, appliance designfacility 108, and manufacturing facility 110.

Dental practitioner 106 may treat patient 102 at clinic 104. Forexample, dental practitioner 106 may create a digital model of thecurrent dental anatomy of patient 102. The dental anatomy may includeany portion of crowns or roots of one or more teeth of a dentalarchform, gingiva, periodontal ligaments, alveolar bone, cortical bone,bone grafts, implants, endodontic fillings, artificial crowns, bridges,veneers, dentures, orthodontic appliances, or any structure (natural orsynthetic) that could be considered part of the dental anatomy ofpatient 102 before, during, or after treatment.

In one example, the digital model of the current dental anatomy includesa three-dimensional (3D) model of the current (pre-treatment) dentalanatomy of patient 102. Clinic 104 may be equipped, in various examples,with an intra-oral scanner, cone beam computed tomography (CBCT)scanning (e.g., 3D X-ray) device, optical coherence tomography (OCT)device, magnetic resonance imaging (MRI) machine, or any other 3D imagecapturing system which dental practitioner 106 may utilize to generatethe 3D model of the dental anatomy of patient 102.

In the example shown in FIG. 1 , clinic 104 is equipped with computingsystem 190. Computing system 190 may represent a single device or asecurely interconnected group of devices. In these examples, theindividual devices of computing system 190 may form the secureinterconnection by being entirely contained within the logical confinesof clinic 104 (e.g., by way of physical connections within clinic 104,such as using a local area network or “LAN”) and/or by way of virtualprivate network (VPN) tunneling-based encrypted communications securelycommunicated over a public network, such as the Internet. Computingsystem 190 may include one or more user-facing computing devices such asa personal computer (e.g., desktop computer, laptop computer, netbook,etc.), mobile device (e.g., tablet computer, smartphone, personaldigital assistant, etc.), or any other electronic device configured toprovide end-user computing capability, such as by presenting resourcesin a human-understandable form (e.g., visual images such asmedical/dental imaging, legible output, symbolic/pictorial output,audible output, haptic output, etc.).

Dental practitioner 106 may store a digital model of the current dentalanatomy of patient 102 to a storage device included in computing system190 or is read/write accessible via computing system 190. In someexamples, computing system 190 may also store a digital model of aproposed dental anatomy for patient 102. The proposed dental anatomyrepresents the intended function, integrity, and morphology of thedental anatomy to be achieved by application of a dental appliance 112as part of dental restoration treatment of patient 102.

In one example, dental practitioner 106 may generate a physical model ofthe proposed dental anatomy and utilize an image capture system (e.g.,as described above) to generate the digital model of the proposed dentalanatomy. In another example, dental practitioner 106 may effectuatemodifications to the digital model of the current anatomy of patient 102(e.g., by adding material to a surface of one or more teeth of thedental anatomy, or in other ways) to generate the digital model of theproposed dental anatomy for patient 102. In yet another example, dentalpractitioner 106 may use computing system 190 to modify the digitalmodel of the current dental anatomy of patient 102 to generate a modelof the proposed dental anatomy for patient 102.

In one scenario, computing system 190 outputs the digital model(s)representing the current and/or proposed dental anatomies of patient 102to another computing device, such as computing device 150 and/orcomputing device 192. Although described herein as being performedlocally at computing systems 190, 150, and 192, it will be appreciatedthat, in some examples, one or more of computing systems 190, 150, and192 may leverage cloud computing capabilities and/or software as aservice (SaaS) capabilities to perform the underlying processing for thefunctionalities described herein. As illustrated in FIG. 1 , in someexamples, computing device 150 of design facility 108, computing system190 of clinic 104, and computing device 192 of manufacturing facility110 may be communicatively coupled to one another via network 114.Network 114 may, in various examples, represent or include a privatenetwork associated with an association (e.g., a dental services network,etc.) or other entity or grouping of entities.

In other examples, network 114 may represent or include a publicnetwork, such as the Internet. Although illustrated as a single entityin FIG. 1 purely for ease of illustration, it will be appreciated thatnetwork 114 may include a combination of multiple public and/or privatenetworks. For instance, network 114 may represent a private networkimplemented using public network infrastructure, such as a VPN tunnelimplemented over the Internet. As such, network 114 may comprise one ormore of a wide area network (WAN) (e.g., the Internet), a LAN, a VPN,and/or another wired or wireless communication network. Network 114 mayinclude a wired or wireless network components that conform to one ormore standards, such as via Ethernet®, WiFi™, Bluetooth®, 3G, 4G LTE,5G, and the like.

In the example of FIG. 1 , computing device 150 is implemented by or atdesign facility 108. Computing device 150 is configured to automaticallydesign a dental appliance and/or generate placement information for thedental appliance for reshaping the dental anatomy of patient 102.Computing device 150 (or components thereof) implement neural networktechnology to determine the geometry of and/or placement information fora dental appliance. In the example shown in FIG. 1 , computing device150 includes one or more processors 172, one or more user interface (UI)devices 174, one or more communication units 176, and one or morestorage devices 178.

UI device(s) 174 may be configured to receive input data from a user ofcomputing device 150 and/or provide output data to a user of computingdevice 150. One or more input components of UI devices 174 may receiveinput. Examples of input are tactile, audio, kinetic, and optical input,to name only a few examples. For example, UI devices 174 may include oneor more of a mouse, keyboard, a voice input system, image capturedevices (e.g., still camera and/or video camera hardware), physical orlogical buttons, a control pad, a microphone or microphone array, or anyother type of device for detecting input from a human user or anothermachine. In some examples, UI devices 174 may include one or morepresence-sensitive input components, such as resistive screens,capacitive screens, single or multi-finger touchscreens,stylus-sensitive screens, etc.

Output components of UI device 174 may output one or more of visual(e.g., symbolic/pictorial or legible) data, tactile feedback, audiooutput data, or any other output data that is intelligible to a humanuser or to another machine. Output components of UI device 174, invarious examples, include one or more of a display device (e.g., aliquid crystal display (LCD) display, touchscreen, stylus-sensitivescreen, a light-emitting diode (LED) display, an optical head-mounteddisplay (HMD), among others), a loudspeaker or loudspeaker array, aheadphone or headphone set, or any other type of device capable ofgenerating output data in a human or machine intelligible format.

Processor(s) 172 represent one or more types of processing hardware(e.g., processing circuitry), such as general-purpose microprocessor(s),specially designed processor(s), an application specific integratedcircuit (ASIC), a field programmable gate array (FPGA), a collection ofdiscrete logic, fixed function circuitry, programmable circuitry (or acombination of fixed function circuitry and programmable circuitry) orany type of processing hardware capable of executing instructions toperform the techniques described herein.

For example, storage device(s) 178 may store program instructions (e.g.,software instructions or modules) that are executed by processor(s) 172to carry out the techniques described herein. In other examples, thetechniques may be executed by specifically programmed circuitry ofprocessor 172 (e.g., in the case of fixed function circuitry orspecifically programmed programmable circuitry). In these or other ways,processor(s) 172 may be configured to execute the techniques describedherein, sometimes by leveraging instructions and other data accessiblefrom storage device(s) 178.

Storage device(s) 178 may store data for processing by processor(s) 172.Certain portions of storage device(s) 178 represent a temporary memory,meaning that a primary purpose of these portions of storage device(s)178 is not long-term storage. Short-term storage aspects of storagedevice(s) 178 may include volatile memory that is not configured toretain stored contents if deactivated and reactivated (e.g., as in thecase of a power-cycle). Examples of volatile memories include randomaccess memories (RAM), dynamic random access memories (DRAM), staticrandom access memories (SRAM), and other forms of volatile memoriesknown in the art. In some examples, short-term memory (e.g., RAM) ofstorage device(s) 178 may include on-chip memory unit(s) collocated withportions of processor(s) 172 to form a portion of an integrated circuit(IC) or a portion of a system on a chip (SoC).

Storage device(s) 178 may, in some examples, also include one or morecomputer-readable storage media. Storage device(s) 178 may be configuredto store larger amounts of data than volatile memory. Storage device(s)178 may further be configured for long-term storage of data asnon-volatile memory space and retain data after activate/off cycles.Examples of nonvolatile memories include, solid state drives (SSDs),hard disk drives (HDDs), flash memories, or forms of electricallyprogrammable memories (EPROM) or electrically erasable and programmable(EEPROM) memories. Storage device(s) 178 may store program instructionsand/or data associated with software components and/or operating systemsof computing device 150.

In the example of FIG. 1 , storage device(s) 178 include appliancefeature library 164, models library 166, and practitioner preferenceslibrary 168 (collectively, “libraries 164-168). Libraries 164-168 mayinclude relational databases, multi-dimensional databases, maps, hashtables, or any other data structure. In one example, models library 166includes 3D models of the patient's current and/or proposed dentalanatomy. In some instances, libraries 164-168 may be stored locally atcomputing device 150 or may be accessed via a networked file share,cloud storage, or other remote datastore accessible using networkinterface hardware of communication unit(s) 176.

Short-term memory of storage device(s) 178 and processor(s) 172 maycollectively provide a computing platform for executing operating system180. Operating system 180 may represent an embedded, real-timemultitasking operating system, for instance, or may represent any othertype of operating system. Operating system 180 provides a multitaskingoperating environment for executing one or more software components182-186. In some examples, operating system 180 may execute any ofcomponents 182-188 as an instance of a virtual machine or within avirtual machine instance executing on underlying hardware. Althoughillustrated separately from operating system 180 as a non-limitingexample, it will be appreciated that any of components 182-188 may beimplemented as part of operating system 180 in other examples.

In accordance with the techniques of this disclosure, computing device150 automatically or semi-automatically generates a digital model ofdental appliance 112 for restoring the dental anatomy of patient 102using one or more types of neural networks trained with dentalanatomy-related data and/or appliance feature data pertaining to patient102 and/or other (actual or hypothetical) patients. Pre-processor 182 isconfigured to pre-process the digital model of the proposed dentalanatomy of patient 102.

In one example, pre-processor 182 performs pre-processing to identifyone or more teeth in the proposed dental anatomy of patient 102. In someinstances, pre-processor 182 identify a local coordinate system for eachindividual tooth and may identify a global coordinate system thatincludes each tooth of the proposed dental anatomy (e.g. in one or botharches of the proposed dental anatomy). As another example,pre-processor 182 may pre-process the digital model of the proposeddental anatomy to identify the root structure of the dental anatomy.

In another example, pre-processor 182 may identify the gingiva of thegums in the proposed dental anatomy, thereby identifying and delineatingportions of the proposed dental anatomy that include gingiva andportions of the proposed dental anatomy that include tooth. As yetanother example, pre-processor 182 may pre-process the digital model ofthe proposed dental anatomy by extending the roots to identify the topsurface of the root of each respective tooth. Pre-processor 182 mayperform one, several, or all of the various example functionalitiesdescribed above, in various use case scenarios, depending on requestsprovided by dental practitioner 106, based on data availability withrespect to patient 102 and/or other patient(s), and potentiallydepending on other factors.

Computing device 150 (or hardware/firmware components thereof) mayinvoke or activate neural network engine 184 to determine placementinformation of dental appliance 112 during dental restorative treatmentof patient 102. In some examples, neural network engine 184 mayimplement a two-hidden-layer neural network trained with placementinformation for patient 102 and/or for other patients with generallycorresponding dental anatomies (current or proposed).

In these examples, neural network engine 184 may implement the neuralnetwork to accept, as inputs, individual position/orientationinformation for each of two teeth (e.g., a pair of adjacent teeth) inthe current dental anatomy of patient 102, and may output placementinformation for dental appliance 112 during dental restorative therapy.For instance, the placement information may directly or indirectlyreflect one or more of the position, orientation, or sizing of dentalappliance 112 as it is to be used in the dental restorative treatment ofpatient 102.

In one example, neural network engine 184 may train the neural networkusing a backpropagation algorithm using a single 4×4 transform for eachof the two adjacent teeth, and another 4×4 transform that identifies a“ground truth” {position, orientation, size} tuple of dental appliance112 after placement is completed as part of the dental restorationtreatment of patient 102. As used herein, the term “transform” refers tochange (or “delta”) information with respect to the {position,orientation, size} tuple, and can therefore also be described as a{translation, rotation, scale} tuple with respect to dental appliance112.

In some instances, the transforms of this disclosure may includeadditional elements as well, such as a shear mapping (or simply “shear”)associated with dental appliance 112. As such, the transforms of thisdisclosure may, in various examples, represent affine transforms, andmay include some or all transformations included in the possibletransformations under the automorphism in an affine space.

Neural network engine 184 may extract the transforms for the specificteeth from 3D mesh data describing the current and/or proposed dentalanatomy of patient 102. As used herein, the term “ground truth” refersto a proven or otherwise well-founded description of dental anatomyfeatures or appliance features. As such, in some examples, a groundtruth transform may be produced manually by dental practitioner 106 or atechnician using a CAD tool.

In other examples, a ground truth transform may be generatedautomatically, such as by using the automated techniques described in WO2020/240351, filed May 20, 2020, the entire content of which isincorporated herein by reference. Various techniques of this disclosureare described below with respect to the non-limiting example ofpositioning, orienting, and sizing of a center clip that is placed overthe gap between two adjacent teeth during dental restoration. It will beappreciated, however, that neural network engine 184 may implement thetechniques of this disclosure to generate geometries and/or placementinformation for other types of dental appliances, as well.

As part of generating the placement information for a center clip,neural network engine 184 may identify landmarks of the proposed dentalanatomy. Example landmarks include a slice, a midpoint, a gingivalboundary, a closest point between two adjacent teeth (e.g., a point ofcontact between adjacent teeth or a point of closest approach (orclosest proximity), a convex hull, a center of mass, or other landmark.A slice refers to a cross section of the dental anatomy. The midpoint ofa tooth refers to a geometric center (also referred to as a geometricalmidpoint) of the tooth within a given slice.

The gingival boundary refers to a boundary between the gingiva and oneor more teeth of a dental anatomy. A convex hull refers to a polygon,the vertices of which include a subset of the vertices in a given set ofvertices, where the boundary of the subset of vertices circumscribes theentire set of vertices. The center of mass of a tooth refers to amidpoint, center point, centroid, or geometric center of the tooth. Insome instances, neural network engine 184 may determine one or more ofthese landmarks as expressed using a local coordinate system for eachtooth.

In some examples, as part of landmark identification, neural networkengine 184 determines a plurality of slices of the patient's proposeddental anatomy. In one example, the thickness of each slice is the same.In some instances, the thickness of one or more slices is different thanthe thickness of another slice. The thickness of a given slice may bepre-defined. In one instance, neural network engine 184 automaticallydetermines the thickness of each slice using the simplified neuralnetwork of this disclosure. In another instance, the thickness of eachslice may be user-defined and, for example, available as a ground truthinput to the simplified neural network.

As part of landmark identification, neural network engine 184 may, insome examples, determine a midpoint for each tooth to which theplacement of dental appliance 112 pertains. In one example, neuralnetwork engine 184 identifies a landmark using a midpoint of aparticular tooth by computing the extrema of that particular tooth'sgeometry based on the entirety of that particular tooth (e.g., withoutdividing the dental anatomy into slices) and determining the midpoint ofthat particular tooth based on the extrema of the tooth geometry.

In some examples, neural network engine 184 may determine a midpoint foreach tooth for each slice. For instance, neural network engine 184 maydetermine the midpoint for a particular slice of a particular tooth bycalculating the center of mass of a constellation of vertices around theedge of the particular tooth for that particular slice. In someinstances, the midpoint of the particular tooth for the particular slicemay be biased toward one edge of the tooth (e.g. in the case that oneedge has more points than another edge).

In other examples, neural network engine 184 may, as part of thelandmark identification portion of the placement generation, determinethe midpoint of a particular tooth in a particular slice based on aconvex hull of the particular tooth for the particular slice. Forexample, neural network engine 184 may determine a convex hull of a setof edge points of the tooth for a given slice. In some instances, neuralnetwork engine 184 executes a neural network that, as part of landmarkidentification, determines a geometric center from the convex hull byperforming a flood-fill operation on the region circumscribed by theconvex hull and computing a center of mass of the flood-filled convexhull.

In some examples, the neural network executed by neural network engine184 outputs a closest point between two adjacent teeth. The closestpoint between two adjacent teeth may be a point of contact or a point ofclosest approach. In one example, neural network engine 184 determines aclosest point between two adjacent teeth for each slice. In anotherexample, neural network engine 184 determines a closest point betweentwo adjacent teeth based on the entirety of the adjacent teeth (e.g.,without dividing the dental anatomy into slices).

Using landmarks computed for the proposed dental anatomy, the neuralnetwork executed by neural network engine 184 generates one or morecustom appliance features for dental appliance 112 based at least inpart on the landmarks. For example, custom feature generator 184 maygenerate the custom appliance features by determining thecharacteristics of the custom appliance features, such as a size, shape,position, and/or orientation of the custom appliance features. Examplesof custom appliance features include a spline, a mold parting surface, agingival trim surface, a shell, a facial ribbon, a lingual shelf (alsoreferred to as a “stiffening rib”), a door, a window, an incisal ridge,a case frame sparing, a diastema matrix wrapping, among others.

In some examples, neutral network engine 184 may identify and usefeatures other than the examples listed above. For example, neuralnetwork engine 184 may identify and use features that are discernibleand actionable to processor(s) 172 within the mathematical framework ofthe executed neural network. As such, the operations performed via theneural network executed by neural network engine 184 may represent a“black box” in terms of the features used and the mathematical frameworkapplied by the neural network during execution.

A spline refers to a curve that passes through a plurality of points orvertices, such as a piecewise polynomial parametric curve. A moldparting surface refers to a 3D mesh that bisects two sides of one ormore teeth (e.g., separates the facial side of one or more teeth fromthe lingual side of the one or more teeth). A gingival trim surfacerefers to a 3D mesh that trims an encompassing shell along the gingivalmargin. A shell refers to a body of nominal thickness. In some examples,an inner surface of the shell matches the surface of the dental arch andan outer surface of the shell is a nominal offset of the inner surface.

The facial ribbon refers to a stiffening rib of nominal thickness thatis offset facially from the shell. A window refers to an aperture thatprovides access to the tooth surface so that dental composite can beplaced on the tooth. A door refers to a structure that covers thewindow. An incisal ridge provides reinforcement at the incisal edge ofdental appliance 112 and may be derived from the archform. The caseframe sparing refers to connective material that couples parts of dentalappliance 112 (e.g., the lingual portion of dental appliance 112, thefacial portion of dental appliance 112, and subcomponents thereof) tothe manufacturing case frame. In this way, the case frame sparing maytie the parts of dental appliance 112 to the case frame duringmanufacturing, protect the various parts from damage or loss, and/orreduce the risk of mixing-up parts.

In some examples, the neural network executed by neural network engine184 generates one or more splines based on the landmarks. The neuralnetwork executed by neural network engine 184 may generate a splinebased on a plurality of tooth midpoints and/or closest points betweenadjacent teeth (e.g., points of contact between adjacent teeth or pointsof closest proximity between adjacent teeth). In some instances, theneural network executed by neural network engine 184 generates onespline for each slice. In one instance, neural network engine 184generates a plurality of splines for a given slice. For instance, neuralnetwork engine 184 may generate a first spline for a first subset ofteeth (e.g., right posterior teeth), a second spline for a second subsetof teeth (e.g., left posterior teeth), and a third spline for a thirdsubset of teeth (e.g., anterior teeth).

Neural network engine 184 generates, in some scenarios, a mold partingsurface based on the landmarks. The mold parting surface may be used tosplit an encompassing shell for molding without undercuts. In someexamples, neural network engine 184 generates additional copies of themold parting surface. For example, the neural network executed by neuralnetwork engine 184 may place one or more copies of a mold partingsurface at small offsets to the main parting surface for the purpose ofcreating an interference condition when the appliance is assembled(which may, for example, improve shape adaptation and sealing whenapplying a tooth restoration material to the teeth).

Appliance feature library 164 includes a set of pre-defined appliancefeatures that may be included in dental appliance 112. Appliance featurelibrary 164 may include a set of pre-defined appliance features thatdefine one or more functional characteristics of dental appliance 112.Examples of pre-defined appliance features include vents, rear snapclamps, door hinges, door snaps, an incisal registration feature, centerclips, custom labels, a manufacturing case frame, a diastema matrixhandle, among others. Each vent is configured to enable excess dentalcomposite to flow out of dental appliance 112.

Rear snap clamps are configured to couple a facial portion of dentalappliance 112 with a lingual portion of dental appliance 112. Each doorhinge is configured to pivotably couple a respective door to dentalappliance 112. Each door snap is configured to secure a respective doorin a closed position. In some examples, an incisal registration featurecomprises a male and female tab pair that falls on the incisal edge ofdental appliance 112 (e.g., along the midsagittal plane). In oneexample, the incisal registration feature is used to maintain verticalalignment of a facial portion of dental appliance 112 and a lingualportion of dental appliance 112.

Each center clip is configured to provide vertical registration betweenthe lingual portion of dental appliance 112 and the facial portion ofdental appliance 112. Each custom label includes data identifying a partof dental appliance 112. The manufacturing case frame is configured tosupport one or more parts of dental appliance 112. For example, themanufacturing case frame may detachably couple a lingual portion ofdental appliance 112 and a facial portion of dental appliance 112 to oneanother for safe handling and transportation of dental appliance 112from manufacturing facility 110 to clinic 104.

The neural network executed by neural network engine 184 may, in someexamples, determine the characteristics of one or more pre-definedappliance features that are included in pre-defined appliance featurelibrary 164. For instance, one or more features accessible frompredefined appliance feature library 164 may represent component shapesobtained in one or more ways, such as by way of manual generation (e.g.,by dental practitioner 106 or via automated generation, such as via thetechniques described in WO 2020/240351, filed May 20, 2020 describedabove). Based on availability and pertinence to the current dentalanatomy of patient 106, the neural network may be trained (at leastpartially) using the information available from pre-defined appliancefeature library 164.

In one example, the pre-defined appliance features are configured toenable or perform functionalities attributed to dental appliance 112.The characteristics of the pre-defined appliance features may includeone or more of the transform-related attributes described above (e.g.,position, orientation, size) and/or other attributes, such as shapeinformation. The neural network executed by neural network engine 184may determine the characteristics of the pre-defined appliance featuresbased on one or more rules, such as rules that are generated and/orrefined via machine learning (ML) techniques.

In some examples, the executed neural network determines placementinformation for a rear snap clamp based on the rules. In one example,neural network engine 184 may generate placement information thatpositions two rear snap clamps along one archform during dentalrestorative treatment, with the two rear snap clamps being disposed onopposite ends of the archform. For instance, a first snap clamp may beplaced at one end of the archform during dental restorative treatment,and a second snap clamp may be placed at the other end of the samearchform during the dental restorative treatment.

In some examples, neural network engine 184 may assign a position to oneor both of the rear snap clamps one tooth beyond the outer-most teeth tobe restored. In some examples, neural network engine 184 positions afemale portion of the rear snap clamp on the lingual side of the partingsurface and position a male portion of the rear snap clamp on the facialside may. In some examples, neural network engine 184 determinesplacement information for a vent during dental restorative treatment,based on the rules. For example, neural network engine 184 may assignthe vent a position at the midline of a corresponding door on an incisalside of dental appliance 112.

In some scenarios, neural network engine 184 determines a placement of adoor hinge based on the rules. In one scenario, neural network engine184 assigns each door hinge a position at the respective midline of acorresponding door. In one scenario, neural network engine 184determines a positioning in which the female portion of the door hingeis anchored to the facial portion of dental appliance 112 (e.g., towardsthe incisal edge of a tooth) and positions the male portion of the doorhinge to anchor to the outer face of the door.

In one instance, neural network engine 184 determines a placement of adoor snap based on the rules by positioning the door snap along amidline of a corresponding door. In one instance, neural network engine184 determines a positioning which the female portion of the door snapanchors to an outer face of the door and extends downward toward thegingiva. In another instance, neural network engine 184 determines aposition according to which the male portion of the door snap isanchored to the gingival side of the facial ribbon. For example, thedoor snap may secure the door in a closed position by latching the maleportion of the door snap to the facial ribbon.

Neural network engine 184 may determine the characteristics of apre-defined appliance feature based on preferences of dentalpractitioner 106. Practitioner preferences library 168 may include dataindicative of one or more preferences of dental practitioner 106. Assuch, neural network engine 184 may use information from practitionerpreferences library 168 that pertains to dental practitioner 106 astraining data in the overall training of the neural network beingexecuted to determine placement or geometry information for dentalappliance 112.

Practitioner preferences may, in various use case scenarios, directlyaffect the characteristics of one or more appliance features of dentalappliance 112. For example, practitioner preferences library 168 mayinclude data indicating a preferred size of various appliance features,such as the size of the vents. In some such examples, larger vents mayenable the pressure of the dental composite or resin to reachequilibration faster during the filling process but may result in alarger nub to finish after curing. In these examples, neural networkengine 184 may train the neural network with scaling information thatsizes dental appliance 112 according to the preference attributed todental practitioner 106.

In other examples, practitioner preferences indirectly affect thecharacteristics of appliance features. For example, practitionerpreferences library 168 may include data indicating a preferredstiffness of the appliance or a preferred tightness of a self-clampingfeature. Such preference selections may also affect more complex designchanges to section thickness of the matrix and or degree of activationof the clamping geometry. Neural network engine 184 may determine thecharacteristics of the appliance features by augmenting the rules uponwhich the implemented neural network is trained using preferences ofdental practitioner 106 (or other dental practitioners, in some cases)available from practitioner preferences library 168. In some examples,neural network engine 184 may augment the rules with the practitionerpreference data based on a simulation (e.g. Monte Carlo) or finiteelement analysis performed using the practitioner preferenceinformation. In some instances, feature characteristics also may bederived from properties in the materials to used with the matrix, suchas type of composite that the dentist prefers to use with the appliance.

Using the outputs of the neural network executed by neural networkengine 184, model assembler 186 generates a digital 3D model of dentalappliance 112 used to re-shape the dental anatomy (e.g., form thecurrent dental anatomy to the proposed dental anatomy) of patient 102.In various examples, model assembler 186 may generate the digital 3Dmodel using custom and/or pre-defined appliance features that form theoutputs of the neural network executed by neural network engine 184. Thedigital 3D model of dental appliance 112 may include, be, or be part ofone or more of a point cloud, 3D mesh, or other digital representationof dental appliance 112. In some instances, model assembler 186 storesthe digital model of dental appliance 112 in models library 166.

Model assembler 186 may output the digital model of dental appliance 112in various ways. In one example, model assembler 186 may output thedigital 3D model of dental appliance 112 to computing device 192 ofmanufacturing facility 110 (e.g., via network 114 using networkinterface hardware of communication unit(s) 176). By providing thedigital 3D model to computing device 192, model assembler 186 may enableone or more entities at manufacturing facility 110 to manufacture dentalappliance 112. In other examples, computing device 150 may send thedigital model of dental appliance 112 to computing system 190 of clinic104. In these examples, model assembler 186 may enable dentalpractitioner 106 or other entities at clinic 104 to manufacture dentalappliance 112 onsite at clinic 104.

In some examples, computing device 192 may invoke network interfacehardware of communication unit(s) 176 to send the digital 3D model ofdental appliance 112 to manufacturing system 194 via network 114. Inthese examples, manufacturing system 194 manufactures dental appliance112 according to the digital 3D model of dental appliance 112 formed bymodel assembler 186. Manufacturing system 194 may form dental appliance112 using any number of manufacturing techniques, such as 3D printing,chemical vapor deposition (CVD), thermoforming, injection molding, lostwax casting, milling, machining, or laser cutting, among others.

Dental practitioner 106 may receive dental appliance 112 and may utilizedental appliance 112 to re-shape one or more teeth of patient 102. Forexample, dental practitioner 106 may apply a dental composite to thesurface of one or more teeth of patient 102 via one or more doors ofdental appliance 112. Dental practitioner 106 or another clinician atclinic 104 may remove excess dental composite via one or more vents.

In some examples, model assembler 186 may store the generate digital 3Dmodel of dental appliance 112 to models library 166. In these examples,models library 166 may provide appliance model heuristics that neuralnetwork engine 184 may use as training data in training one or moreneural networks. In some examples, models library 166 includes dataindicative of appliance success criteria associated with each completedinstance of dental appliance 112. Neural network engine 184 may augmentthe neural network training datasets with any appliance success criteriaavailable from models library 166. The appliance success criteria mayindicate one or more of a manufacturing print yield, practitionerfeedback, patient feedback, customer feedback or ratings, or acombination thereof.

For example, neural network 184 may train the neural network to generatea new or updated placement profile and/or geometry of dental appliance112 via the digital 3D model using the appliance success criteriadetermined for previously generated dental appliances. The neuralnetwork executed by neural network engine 184 may, as part of thetraining, determine whether the appliance success criteria meet one ormore threshold criteria, such as one or more of a thresholdmanufacturing yield, a threshold practitioner-provided rating, athreshold patient satisfaction rating, etc.

In one example, the existing digital 3D model available from modelslibrary 166 is a template or reference digital model. In such examples,neural network engine 184 may train the neural network partly based onthe template digital model. The template digital model may, in variousexamples, be associated with different characteristics of the currentdental anatomy of patient 102, such as a template for patients havingsmall teeth or impediments to opening the mouth beyond a certain width.

In one example, neural network engine 184 trains the neural networkusing previously generated digital 3D models available from modelslibrary 166. For example, neural network engine 184 utilize one or moremorphing algorithms to adapt the previously generated digital 3D modelsaccessed from models library 166 to the situation represented by thedental restorative treatment being tailored to the dentition of patient102 during training and/or execution of the neural network.

For example, neural network engine 184 may utilize morphing algorithmsto interpolate appliance feature geometries, and/or may generate a newdigital model of a dental appliance 112 based on the design of theexisting digital model. In one instance, the design feature of anexisting digital model may include a window inset from the perimeter,such that neural network engine 184 may morph the geometry of theexisting digital model based on landmarks for a different dentalanatomy.

Neural network engine 184 trains and executes the neural network toperform (and potentially, compress) multiple intermediate steps in theprocess of generating a digital 3D model of dental appliance 112 forplacement and geometry purposes. Neural network engine 184 generates thefeature set describing dental appliance 112 using 3D meshes of thecurrent and/or proposed dental anatomy of patient 102. The 3D meshes (or“tooth meshes”) and, in examples where available, library components,form training inputs to the neural network.

As described above, neural network engine 184 may train the neuralnetwork to automate one or both of component placement and/or geometrygeneration for components of dental appliance 112. Examples ofcomponents include a center clip registration tab (or “beak”), doorhinges, door snaps, door vents, rear snap clamps, and various others.Examples of placement-related factors and/or components that neuralnetwork engine 184 may generate include a parting surface, a gingivaltrim, doors, windows, facial ribbon, incisal ridge, lingual shelf,diastema matrix, case frame, part label(s), etc.

Neural network engine 184 implements the neural network to automateplacement information generation and/or geometry-generation operationsfor components such as door hinges and center clips, must be placed inspecific positions relative to 3D representations of the teeth of thedental anatomy of patient 102 to perform dental restoration. Byleveraging neural network technology to automate the placement and/orgeometry information of these components, significantly reduces theprocess turnaround for using dental appliance 112 during dentalrestorative therapy of patient 102.

Also, by leveraging a neural network trained with the combination ofdatasets set forth in this disclosure, neural network engine 184automates the placement and/or geometry generation with improvedconsistency and precision, such as by updating the training of theneural network based on ongoing feedback information or otherdynamically changing factors.

The neural network-based automated algorithms of this disclosureimplemented by computing device 150 provide several advantages in theform of technical improvements in the technical field of restorativedental appliance configuration. As one example, computing device 150 mayinvoke neural network engine 184 to generate placement and/or geometryinformation for dental appliance 112 without the need to explicitlycompute tooth geometry landmarks at each instance.

Instead, neural network engine 184 may execute a neural network trainedwith the various transform data and/or ground truth data described aboveto generate the placement and/or geometry information based on thesetraining factors. As another example, computing device 150 may improvethe data precision associated with the digital 3D model of dentalappliance 112 by continuously improving the output of the neural networkbased on treatment plans and results from previous patients, feedbackfrom dental practitioner 106 and/or patient 102, and other factors thatcan be used to fine tune the neural network using ML techniques.

As another example, because neural network engine 184 may implementfurther refinements to the algorithms through introducing new trainingdata rather than modifying rules-based logic, the techniques of thisdisclosure may provide reusability and compute resource sustainabilityimprovements, as well. While described primarily with respect toappliances used in dental restorative therapy as an example, it will beappreciated that neural network engine 184 may, in other examples, beconfigured to implement algorithms of this disclosure to generategeometry and/or placement information for other types of dentalappliances as well, such as orthodontic equipment, surgical guides, andbracket bonding templates, etc.

While computing device 150 is described herein as performing both thetraining and execution of the various neural networks of thisdisclosure, it will be appreciated that, in various use case scenarios,the training of the neural networks may be performed by devices orsystems that are separate from the devices that execute the trainedneural networks. For instance, a training system may use some or all ofthe training data described with respect to FIG. 1 , in the form oflabeled training datasets, to form one or more trained models. Otherdevices may import the trained model(s), and execute the trainedmodel(s) to produce various neural network output(s) described above.

FIG. 2 is a flowchart illustrating an example process 200 that system100 may perform to generate a digital model of a dental appliance byexecuting a neural network trained according to aspects of thisdisclosure. Process 200 may begin with training phase 201. As part oftraining phase 201, neural network engine 184 may train the neuralnetwork using transform data associated with 3D models of various dentalanatomies (202). According to various aspects of this disclosure, neuralnetwork engine 184 may train a neural network using backpropagationtraining techniques.

In some examples, neural network engine 184 may use one 4×4 transformfor each of one or more teeth of the dental anatomy, and one 4×4transform that defines the ground truth {position, orientation, size}tuple for dental appliance 112 after placement is completed. In theexample of dental appliance 112 representing a center clip, neuralnetwork engine 184 may extract the transforms for the pair of maxillarycentral incisors (denoted as teeth “8 and 9” in the universal notationsystem for permanent teeth) or the pair of mandibular central incisors(denoted as teeth “24 and 25” in the universal notation system forpermanent teeth) from the 3D mesh data describing the proposed dentitionof patient 102.

The ground truth transform may represent “pristine” data that isproduced manually by a technician using computer-aided design (CAD) toolto position and orient the center clip, or alternatively, may representpristine data produced automatically in various ways, such as by usingthe automated techniques described in WO 2020/240351, filed May 20,2020. By training the neural network using the backpropagationalgorithm, neural network engine 184 generates multiple layers that arefully connected. That is, a weighted connection exists between a givennode in a first layer and each of respective node in the next layer.

The backpropagation algorithm implemented by neural network engine 184adjusts the weights for these inter-layer node-to-node connectionsthrough the course of training the neural network, thereby graduallyencoding the desired logic into the neural network over the course ofmultiple training iterations or passes. In some examples of thisdisclosure, the neural network may include two layers, therebymitigating computational overhead with respect to both training andeventual execution.

While described herein with respect to training the neural network withtransforms for two teeth using data from one or more past cases, it willbe appreciated that, in other examples, neural network engine 184 maytrain the neural network with different types and/or quantities oftraining data as well. The augmentation of training data may depend onthe availability and accessibility of such training data. For example,if accessible from models library 166 or from another source, neuralnetwork engine 184 may augment the training data with which to train theneural network using transforms for other teeth of the archform to whichdental appliance 112 is to be applied, and/or using transforms for oneor more teeth of the opposing archform.

In this way, neural network engine 184 may train the neural networkusing training data that enables the neural network to determinepositioning information for dental appliance 112 based on a moreholistic evaluation of the dental anatomy of patient 102. In theseand/or other examples, neural network engine 184 may augment thetraining data with pertinent preference information available frompractitioner preferences library 168, with patient feedback information,and/or with various other pertinent data that is accessible to computingdevice 150.

After completing training phase 201 of process 200, neural networkengine 184 may commence execution phase 203. Execution phase 203 maybegin when computing device 150 receives a digital 3D model of aproposed (e.g., post dental restoration therapy) dental anatomy forpatient 102 (204). In one example, computing device 150 receives thedigital 3D model of the proposed dental anatomy from another computingdevice, such as computing system 190 of clinic 104. The digital 3D modelof the proposed dental anatomy for patient 102 may include a point cloudor 3D mesh of the proposed dental anatomy.

A point cloud includes a collection of points that represent or definean object in a 3D space. A 3D mesh includes a plurality of vertices(also referred to as points) and geometric faces (e.g., triangles)defined by the vertices. In one example, dental practitioner 106generates a physical model of the proposed dental anatomy for patient102, and utilizes an image capture system to generate the digital 3Dmodel of the proposed dental anatomy from images captured of thephysical model. In another example, dental practitioner 106 modifies adigital 3D model of the current dental anatomy of patient 102 (e.g., bysimulating the addition of material to a surface of one or more teeth ofthe dental anatomy, or by simulating other changes) to generate thedigital 3D model of the proposed dental anatomy. In yet another example,computing system 190 may modify the digital model of the current dentalanatomy to generate a model of the proposed dental anatomy.

In some examples, pre-processor 182 pre-processes the 3D model of theproposed dental anatomy to generate a modified model by digitallyextending the roots of the initial digital model of the proposed dentalanatomy according to the proposed root extension determined bypre-processor 182, thereby more accurately modeling the complete anatomyof the patient's teeth (206). Step 206 is illustrated in FIG. 2 using adashed-line border, to indicate the optional nature of step 206. Forexample, the pre-processing functionalities provided by step 206 may, insome use case scenarios, be subsumed by the functionalities describedherein with respect to neural network engine 184.

In some examples in which pre-processor 182 performs step 206, becausethe tops (e.g., the area furthest from the gingival emergence) of theroots may be at different heights, pre-processor 182 may detect thevertices corresponding to the tops of the roots and then project thosevertices along a normal vector, thereby digitally extending the roots.In one example, preprocessor 182 groups vertices into clusters (e.g.,using a k-means algorithm). Pre-processor 182 may compute the averagenormal vector for each cluster of vertices.

For each cluster of vertices, pre-processor 182 may determine a sum ofresidual angular differences between the average normal vector for thecluster and the vector associated with each of the vertices in thecluster. In one example, pre-processor 182 determines which cluster ofvertices is the top surface of the roots based on the sum of theresidual angular differences for each cluster. For example,pre-processor 182 may determine that the cluster with the lowest sum ofresidual angular differences defines the top surface of the roots.

Neural network engine 184 may obtain one or more tooth transforms basedon the proposed dental anatomy represented in the received 3D model(208). For example, neural network engine 184 may extract respective{translation, rotation, scaling} tuples for one or more teethrepresented in the 3D model based on corresponding {position,orientation, size} tuples for the teeth in the current dental anatomyand dental restoration outcome information shown in the 3D model of theproposed (post-restoration) dental anatomy for patient 102.

Neural network engine 184 may execute the trained neural network tooutput placement information for dental appliance 112 (210). In theexample of dental appliance 112 representing a center clip, neuralnetwork 184 may input two teeth transforms (e.g. transforms describingthe positions, orientations, and dimensionality of the two adjacentmaxillary central incisors) to the neural network with two layersdescribed above. The neural network executed by neural network 184 mayoutput a transform that positions the center clip (which may represent alibrary part) between the two maxillary central incisors, and orientednormal to the overall archform of a current, intermediate, or proposeddental anatomy of patient 102.

Neural network engine 184 may generate the transform for dentalappliance 112 as applied to the dental restorative treatment of patient102 (the output of the trained neural network upon execution) usingvarious underlying operation sets. As one example, the trained neuralnetwork may process the 3D model of the proposed dental anatomy ofpatient 102 to automatically detect a set of one or more landmarks ofthe proposed dental anatomy. In this example, each “landmark” representsan identifiable geometric construct within the 3D model that is usefulfor determining the position and orientation with respect to one or moretooth surfaces. In some examples, the landmarks computed by the trainedneural network include one or more slices of the dental anatomy, whereeach slice may include one or more additional landmarks. For example,the trained neural network may divide the 3D mesh of the proposed dentalanatomy into multiple slices, and may compute one or more landmarks foreach slice, such as a midpoint for each tooth in the slice, a closestpoint between two adjacent teeth (e.g., a point of contact between twoadjacent teeth or a point of closest approach between two adjacentteeth), a convex hull for each tooth in the slice, among others.

Model assembler 186 generates a 3D model of dental appliance 112 (212).In various examples of this disclosure, model assembler 186 mayconstruct an overall 3D mesh for dental appliance 112 based on placementinformation indicated by the transform output by the neural networkexecuted by neural network engine 184. For example, model assembler 186may generate the overall 3D mesh for dental appliance 112 based on oneor more of the placement characteristics indicated by the {translation,rotation, scale} tuple of a transform of this disclosure.

In some examples, model assembler 186 may also factor shear or shearmapping information for dental appliance 112 into the output transform,if shear information is available for the input data to the executedneural network and/or if neural network engine 184 otherwise generatesshear information for dental appliance 112 by executing the trainedmodel of the neural network.

In one example, model assembler 186 may extrapolate (e.g., viaintegration or other similar techniques) one or more characteristics ofa {position, orientation, size} tuple for dental appliance 112 from thetransform output by the neural network. The {position, orientation,size} tuple of each 3D mesh generated by model assembler 186 correspondsto a set of appliance features (e.g., one or both of custom and/orpre-defined appliance features) for the proposed overall structure ofdental appliance 112. In one example, model assembler 186 may determinethe position of a custom appliance feature based on the midpoint of aparticular tooth.

For example, modeler assembler 186 may align or otherwise position a 3Dmesh of a window and/or door (as example features) based on a midpointof the tooth. In this way, model assembler 186 may determine theposition of a pre-defined appliance feature based on the transforminformation output by the neural network executed by neural networkengine 184. As one example, model assembler 186 may determine theposition of a rear snap clamp based on the position of the teeth in thecurrent dental anatomy of patient 102.

In some instances, model assembler 186 determines the position of apre-defined appliance feature based on the position of a customappliance feature. For instance, modeler assembler 186 may align a doorhinge, door snap, and/or vent with a midline of a door. Further, themodel assembler 186 may adjust the feature orientation, scale, orposition based on analysis of the overall model, such as performing afinite element analysis to adjust the active clamping forces of snapclamp. Model assembler 186 may also make adjustments (e.g. to fine tune)based on subsequent expected manufacturing tolerances, such as providingsuitable clearance between features.

Similarly, model assembler 186 may make adjustments based on theproperties of the material used in the creation of the physicalappliance, such as increasing thicknesses when using more flexiblematerials. In various examples in accordance with this aspects of thisdisclosure, model assembler 186 may generate the digital 3D model ofdental appliance 112 to include one or more of a point cloud, a 3D mesh,or other digital representation(s) of dental appliance 112.

Computing device 150 outputs the digital 3D model of dental appliance112 (214). For example, computing device 150 may output the digital 3Dmodel of dental appliance 112 to computing device 192 of manufacturingfacility 110 by invoking network interface hardware of communicationunit(s) 176 to transmit packetized data over network 114. Manufacturingsystem 194 manufactures dental appliance 112 (216). For instance,computing device 192 may control manufacturing system 194 to fabricatedental appliance 112 such that it complies with the placementinformation generated by the trained neural network executed by neuralnetwork engine 184 (e.g., based on the digital 3D model of dentalappliance 112 generated by model assembler 186). In various examples,manufacturing system 194 may generate the physical dental appliance 112via 3D printing, CVD, machining, milling, or any other suitabletechnique.

In some examples, computing system 150 receives feedback on dentalappliance 112 from dental practitioner 106 (218). The optional nature ofstep 218 is illustrated in FIG. 2 by way of dashed-line borders. Forexample, after dental practitioner 106 receives the physical dentalappliance 112 and uses it for dental restorative treatment of patient102, dental practitioner 106 may utilize computing system 190 to sendfeedback to computing device 150. As one example, computing device 150may receive data indicating a request to adjust a characteristic (e.g.,size, positioning characteristics, orientation characteristics, etc.) offuture dental appliances designed according to transform data output bythe neural network for the general patient cohort of patient 102.

In some examples, computing system 150 updates practitioner preferenceslibrary 168 based on the received practitioner feedback (220). Theoptional nature of step 220 is illustrated in FIG. 2 by way ofdashed-line borders. In some examples, neural network engine 184 may usedata available from practitioner preferences library 168 (updated on anongoing basis using incoming feedback from practitioners) to train theneural network on an ongoing basis.

FIG. 3 is a flow diagram illustrating an example use of a neural networkfor placement of a library component of dental appliance 112, inaccordance with aspects of this disclosure. FIG. 3 is described withrespect to the example of using a two-hidden-layer neural network todetermine placement of a center clip registration tab at a specifiedposition and orientation relative to the two maxillary central incisors(“teeth 8 and 9” as denoted in the universal notation system forpermanent teeth). The center clip's origin is to be placed atapproximately the midpoint of the two maxillary central incisors, and tobe oriented such that the center clip's vertical (or ‘Y’) axis is normalto the archform in which the maxillary incisors are positioned.

Neural network engine 184 may implement a backpropagation-based trainingof the neural network, using one 4×4 transform for each of the maxillarycentral incisors, and one 4×4 transform that defines the ground truthposition and orientation of a post-placement center clip. Neural networkengine 184 may extract the transforms for the maxillary central incisorsfrom the 3D mesh data that describe the current dentition of patient192. Neural network engine 184 may obtain the ground truth transformfrom various sources, such as manual production by a technician using aCAD tool to position and orient the center clip, or alternatively, viaautomatic generation, such as by using the techniques described in WO2020/240351, filed May 20, 2020.

In the example of FIG. 3 , neural network engine 184 converts each ofthe 4×4 transforms for the maxillary central incisors to a respective1×7 quaternion vector. Neural network engine 184 concatenates these two1×7 quaternion vectors to yield a single 1×14 feature vector. The 1×14feature vector corresponds to a single patient's data (hereinafter, asingle “case”). An ‘n’ by 14 matrix may be formed by laterallyconcatenating the feature vectors for ‘n’ cases, where ‘n’ denotes anonnegative integer value.

In this way, neural network engine 184 may encode data for multiplecases into a single matrix that can be used as a training input to trainthe two-hidden-layer neural network of FIG. 3 . Neural network engine184 may use the backpropagation algorithm to train the neural network insome non-limiting examples of this disclosure. The layers of the neuralnetwork are fully connected, meaning that a weighted connection existsbetween a node i in a first layer, and each of the nodes j in the nextlayer.

The backpropagation training algorithm executed by neural network engine184 adjusts these weights through the overall course of training (e.g.,through multiple iterations or passes of training and fine tuningthereof) to gradually encode the desired logic into the neural networkover the course of multiple training iterations/passes. According to thespecific example illustrated in FIG. 3 , neural network engine 184 usesa fully connected feedforward neural network with two hidden layers.With FIG. 3 being read from left to right, the first and second hiddenlayers have dimensions of 1×32 and 1×64 respectively, and the outputdimensions are 1×7.

In other examples consistent with the techniques of this disclosure,neural network engine 184 may utilize other neural network architecturesand techniques, such as a Recurrent Neural Network (RNN), restrictedBoltzmann machine (RBM), Long Short-Term Memory (LSTM), ConvolutionalNeural Network (CNN) technology, and various others. In other examples,such as those in which neural network engine 184 uses 3D meshes asinputs, neural network engine 184 may use a Graph CNN to generateplacement information for one or more library components, such as thecenter clip discussed above.

In other examples still, neural network engine 184 may use an image ofthe maxillary central incisors (e.g., an image produced from a rendercould be taken of the two teeth) as an input to a CNN or a fullyconnected neural network to produce the placement transform of thelibrary component. Neural network engine 184 passes the 14 input nodes'values along the weighted connections to the first hidden layer of FIG.3 . Neural network engine 184 passes each node value in the first hiddenlayer (with weighting factors in applicable scenarios) to each of thecorresponding nodes in the second hidden layer, and so on.

Once the inputs have finished propagating through the neural network,neural network 184 converts the resulting 1×7 vector (interpreted as aquaternion) to a 4×4 matrix, which represents the predicted clipplacement matrix. Neural network 184 then computes the layernormalization for the second layer (the “L2 Norm”) of the differencebetween the placement matrix and the ground truth transform that wasinput as a 4×4 transform. The resulting difference represents a “loss”value, which neural network engine 184 may feed back into the neuralnetwork to update the weights via backpropagation.

After multiple iterations of this backpropagation process, the neuralnetwork is trained to take, as inputs, two teeth transforms (i.e.describing the positions and orientations of two teeth) and output atransform that positions a library part (e.g. the center clip) inbetween those teeth, and oriented normally to the archform. The programflow for the training of this particular example is shown in FIG. 3 .Although FIG. 3 is described primarily with respect to the example ofusing a two-hidden-layer neural network, neural network engine 184 maytrain and execute more complex computational models in other examples,such as those of deep learning (e.g. a generative adversarial network or“GAN”) in accordance with aspects of this disclosure.

FIG. 4 is a flow diagram illustrating an example of neural network-basedcomponent geometry generation in accordance with aspects of thisdisclosure. Neural network engine 184 may train the neural network-basedsystem of FIG. 4 to generate custom component(s), such as dentalappliance 112 or discrete components thereof. In some examples, neuralnetwork engine 184 may generate a component, such as a mold partingsurface using a Graph CNN. FIG. 4 is described herein with respect toimplementing a CNN as the generator network component of a GAN.

In the examples described herein with respect to FIG. 4 , the dentitionof patient 102 (or of patients of prior cases) is described by a set of3D meshes, with each tooth being represented by its own individual 3Dmesh. If the transform of each tooth is suitably positioned and orientedto reflect that particular tooth's location in the archform, then each3D mesh includes a list of vertices and a list of faces which describethe relationships between the vertices. In other words, each 3D mesh mayspecify which vertices are a part of which face.

In these examples, each face is a triangle. Neural network engine 184inputs the 3D tooth meshes to the Graph CNN shown in FIG. 4 . The GraphCNN, in turn, generates a component geometry as the output. The outputrepresents the generated component in the form of another 3D meshcomprising respective vertices and faces. The Graph CNN may generatethis 3D mesh in one of two pathways, namely 1) by generating a new setof vertices that describe the generated component; or 2) by moving a setof pre-existing vertices.

In the second technique (which involves moving a set of pre-existingvertices), the generator Graph CNN may start from a template or ageneralized example of the generated component, and then manipulate thestarting set of vertices to make the generated component conform to thedentition (e.g., current and/or intermediate in-treatment dentalanatomy) of patient 102. In turn, neural network engine 184 feeds thepairing of the 3D mesh representing the component generated by the GraphCNN and the 3D tooth mesh originally input to the Graph CNN into adifferentiable discriminator network.

Neural network engine 184 also feeds a second pairing into thedifferentiable discriminator network, namely, the pairing of the groundtruth library component and the 3D tooth mesh originally input to theGraph CNN. The differentiable discriminator network computes theprobability that the input pair came from the second dataset (i.e., thepairing that included the ground truth generated component). That is,the differentiable discriminator network computes the probability ofeach input dataset corresponds to a ground truth dataset including theoriginal tooth meshes and the ground truth mesh of the target geometry.

The differential discriminator network produces gradients, which neuralnetwork engine 184 may use as a loss function for the generator network(in this case, implemented as a CNN) illustrated in FIG. 4 . In thecontext of machine learning, a loss function quantifies the extent towhich a machine learning model differs from an ideal model, and the lossfunction is used to guide the training of the machine learning model.The generator network may also use other loss functions, such asnormalization of individual layers (e.g., the L1 Norm and/or the L2norm) and the Chamfer distance (which is a sum of positive distancesdefined for unsigned distance functions). In other examples, neuralnetwork engine 184 may input an image of the maxillary central incisors(e.g., an image produced from a render taken of the two teeth) to a CNNor to a fully connected neural network to produce a mesh of the groundtruth component.

In various examples, the ground truth generated components may beproduced manually using a CAD tool, or automatically using thetechniques described in WO 2020/240351, filed May 20, 2020. In someexamples, subject to availability, neural network engine 184 may augmentthe training of the generator network using placement information fordental appliance 112, such as the transform output by the neural networkof FIG. 2 .

Although discussed primarily with respect to dental appliances (such asdental appliance 112) used in dental restorative treatment as anexample, it will be appreciated that the neural network-based placementand/or geometry generation techniques of this disclosure can also beused with respect to other types of dental appliances. Non-limitingexamples of other dental appliances that computing device 150 cancustomize using the techniques of this disclosure include a lingualbracket, an overall lingual bracket system, an orthodontic aligner(e.g., a transparent aligner or clear aligner), a bonding tray, etc. Forinstance, neural network engine 184 may train a GAN generator such asthe Graph CNN generator of the GAN of FIG. 4 , but using ground truthgenerated components used in these other types of dental appliances. Inthese examples, the trained generator Graph CNN may produce generatedfeatures for any of these other types of dental appliances, such as oneor more mold parting surfaces. Examples of a geometry that neuralnetwork engine 184 may use to generate features of a lingual bracketsare discussed below with reference to FIGS. 7 & 8 .

For example, computing device 150 may invoke neural network 184 togenerate placement and/or geometry information for a lingual bracketsystem that would otherwise be designed by technicians using customsoftware. As part of the design of the lingual bracket system, neuralnetwork engine 184 may generate specifications for a bonding pad for aspecific tooth. Neural network engine 184 may train the neural networkto subsume various steps of the custom software-based generationprocess, such as outlining a perimeter on the specific tooth,determining a thickness to form a shell, and subtracting out thespecific tooth via a Boolean operation.

Neural network engine 184 may train the neural network to select bracketbodies from a library (e.g., appliance feature library or models library166), virtually place the selected bracket body on the pad, and unitethe pad and the bracket body mounted on it via Boolean additionoperations. The neural network may adjust one or more bracket components(e.g. hooks, wings, etc.) to adapt the overall bracket to the particulargeometry of the specific tooth and the adjacent gingiva. The neuralnetwork, when executed by neural network engine 184, may generate adesign according to which the adjusted bracket component is united withthe bracket body to complete the digital design of the overall bracket,and may export the overall bracket geometry.

Neural network engine 184 may encode the overall bracket geometry invarious ways for export, such as in the form of a stereolithography (SU)file that stores 3D geometry information. To train the neural networkfor lingual bracket generation, neural network engine 184 may leveragepast cases for a patient cohort. With a number of past cases beingavailable for a variety of patients, neural engine 184 may train theneural network to implement automated design of lingual brackets withrelatively little (or no) retraining, thereby conserving computeresource overhead, and with improved accuracy for specific dentalanatomy idiosyncrasies of patient 102, thereby providing improved dataprecision.

Examples of custom appliance features that the generator network of theGAN may generate include 3D mesh-represented information for a spline, amold parting surface, a gingival trim surface, a shell, a facial ribbon,a lingual shelf, a door, a window, among others. In one example, thegenerator network may generate one or more digital meshes representingsplines for each slice of the dental anatomy. The generator network ofthe GAN may generate a spline for a given slice based on a plurality oftooth midpoints of teeth within the slice and/or closest points betweenadjacent teeth within the slice (e.g., points of contact betweenadjacent teeth within the slice or points of closest proximity betweenadjacent teeth within the slice). In other words, in this example, thegenerator network accumulates a set of points (e.g., tooth midpoints,points of contact between adjacent teeth, points of closest approachbetween adjacent teeth, or a combination thereof) for each slice togenerate features representing a spline for each digital slice.

In some examples, the generator network generates a mold parting surfaceas one example feature to be incorporated within an overall 3D model ofa dental restoration appliance. Neural network engine 184 may executethe generator network to generate the mold parting surface based on theplurality of midpoints and/or closest points between adjacent teeth. Forexample, the generator network may accumulate a plurality of the pointsfor each spline for each slice to generate the mold parting surface. Asone example, in an example where the generator network divides thedental anatomy into four slices and generates a single spline for eachslice, the points of each of the four splines may be aggregated togenerate the mold parting surface.

In one scenario, neural network engine 184 may feed preferenceinformation for dental practitioner 106 from practitioner preferenceslibrary 168 into the ground truth repository, to be used as trainingdata augmentation. For instance, neural network engine 184 may querypractitioner preferences library 168 to determine preferences for dentalpractitioner 106. Examples of data stored within practitionerpreferences library 168 include a preferred size, positioning, ororientation of a pre-defined appliance feature for dental practitioner106.

Neural network engine 184 may also train the generator network usingdata indicative of pre-defined appliance features, such as by accessingand retrieving the data from one or more libraries (e.g., as stored in adatastore, database, data lake, file share, cloud repository or otherelectronic repository) of 3D meshes representing pre-defined featuresfor incorporation within an overall 3D model of dental appliance 112.For example, neural network engine 184 may receive these data byquerying appliance feature library 164. Appliance feature library 164stores data defining 3D meshes for a plurality of pre-defined appliancefeatures, such as vents, rear snap clamps, door hinges, door snaps, anincisal registration feature (also referred to as a “beak”), amongothers.

In one example, neural network engine 184 selects one or morepre-defined appliance features of a plurality of pre-defined appliancefeatures stored within appliance feature library 164. For example,appliance feature library 186 may include data defining a plurality ofdifferent predefined appliance features of a given type of pre-definedappliance feature. As one example, appliance feature library 164 mayinclude data defining different characteristics (e.g., size, shape,scale, orientation) for a given type of pre-defined appliance feature(e.g., data for differently sized and/or differently shaped hinges,etc.). In other words, appliance feature library 164 may determine thecharacteristics of a pre-defined appliance feature and select a featurefrom the predefined appliance library that corresponds to the determinedcharacteristics.

In some scenarios, neural network engine 184 selects, for training dataaugmentation, a pre-defined appliance feature (e.g., a particularlysized door hinge) from appliance feature library 164 based on landmarksfor a corresponding tooth, characteristics (e.g., size, type, location)of the corresponding tooth (e.g., a tooth for which the appliancefeature will be used to restore when the dental appliance is applied tothe patient), practitioner preferences, or both.

In other examples, appliance feature library 164 includes data defininga set of required pre-defined appliance features. In some such examples,neural network engine 184 may retrieve data for the 3D meshesrepresenting the pre-defined features for each of the requiredpre-defined features to use as additional training data. In suchexamples, the generator network of the GAN may transform the 3D mesh forincluding in the patient specific dental appliance. For example, thegenerator network may rotate or scale (e.g., re-size) a 3D mesh for aparticular feature based on the landmarks for a corresponding tooth,characteristics of the tooth, and/or practitioner preferences.

FIG. 5 is a flowchart illustrating process 500 that computing device 150may implement to generate component geometries using a GAN, according toaspects of this disclosure. Process 500 generally corresponds totechniques described above with respect to FIG. 4 . Process 500 maybegin with training phase 501, in which neural network engine 184obtains 3D meshes of ground truth dental appliance component geometries(502). In various examples, neural network engine 184 may obtain the 3Dmeshes of the ground truth dental appliance component geometries fromsources that provide manually generated component geometries, or sourcesthat provide component geometries automatically generated using thetechniques described in WO 2020/240351, filed May 20, 2020.

Also as part of training phase 501, neural network engine 184 may traina generator network (e.g., the Graph CNN of FIG. 4 ) with the groundtruth component geometries and 3D tooth meshes using a discriminatornetwork (504). For example, neural network engine 184 may train thegenerator network by feeding {generated component geometry, 3D toothmesh} pairs and {ground truth component geometry, 3D tooth mesh} pairsinto the discriminator network. Neural network engine 184 may executethe discriminator network to calculate probabilities for each pairing,indicating whether or not the respective pairing is based on a groundtruth component geometry.

While step 504 is illustrated in FIG. 5 as a single step purely for easeof illustration, it will be appreciated that neural network engine 184runs the discriminator network in multiple iterations to train thegenerator network and continually fine-tune the training until thegenerator network generates component geometries that are sufficientlyaccurate to “spoof” ground truth geometry-based pairings with respect tothe discriminator network.

Once neural network engine 184 determines that the generator network issufficiently trained, neural network engine 184 may temporarily shelveor potentially even permanently discard the discriminator network forexecution phase 503 of process 500. To begin execution phase 503, neuralnetwork engine 184 may execute the trained generator network to generatea component geometry using 3D tooth meshes of the current dental anatomyof patient 102 as inputs (506).

In one non-limiting example, neural network engine 184 may execute thetrained generator network to generate a mold parting surface of dentalappliance 112 by executing the trained generator network. In turn,manufacturing system 194 manufactures dental appliance 112 according tothe component geometry generated by the trained generator network (508).For example, computing device 150 may output a 3D mesh of the componentgeometry generated by neural network engine 184 to computing device 192of manufacturing facility 110 by invoking network interface hardware ofcommunication unit(s) 176 to transmit packetized data over network 114.

FIG. 6 is a rendering illustrating an example center clip placementperformed according to the neural network-based placement techniques ofthis disclosure discussed above with respect to FIGS. 2 and 3 . In thetwo views illustrated in FIG. 6 , the center clip is placed between(e.g., centered at or substantially at the midpoint of) the twomaxillary central incisors (teeth 8 and 9 according to the universalnumbering system for permanent teeth) and oriented to be normal to thearchform in the proposed dental anatomy of patient 102.

FIG. 7 is a rendering illustrating an example of a bonding pad (e.g., ofa lingual bracket) that is customized to the shape of the correspondingtooth. As described above, the GAN-based techniques described withrespect to FIGS. 4 & 5 may be used to generate the geometry of suchbonding pads.

FIG. 8 is a rendering illustrating an example of a set of componentsthat make up a lingual bracket. The techniques described above withrespect to FIGS. 1-5 may be used to assemble brackets such as theoverall bracket shown in FIG. 8 , and/or to generate placementinformation for the bracket on the teeth of patient 102.

FIG. 9 is a flow diagram illustrating another example of neuralnetwork-based component geometry generation in accordance with aspectsof this disclosure. Neural network engine 184 may train and execute thegenerator network of the GAN shown in FIG. 9 to refine or fine-tune apreviously generated dental appliance model to form an updated dentalappliance model or updated model of a component thereof. Neural networkengine 184 may refine (or fine-tune or “tweak”) the geometry of dentalappliance models that are automatically generated using landmarkinformation (e.g., using the techniques described in WO 2020/240351,filed May 20, 2020) or the geometry of dental appliance models generatedmanually using a computer-aided design (CAD) tool to form the updatedmodel (e.g., updated 3D) of this disclosure.

The GAN generator-based refinement of a previously generated modelprovides a time reduction, and in many instances, an improvement inaccuracy and data precision with respect to geometry modificationsrequired to render the dental appliance model viable (the updated modelrepresenting the viable dental appliance component for use in dentalrestoration treatment). Upon being trained to sufficiently spoof thediscriminator network and/or to pass visual inspection, the generatornetwork is configured to gradually modify component design to bring theupdated model of the dental appliance geometry in line with a designthat is usable during dental restoration treatment of patient 102.

In contrast to the techniques described with respect to FIG. 4 (in whicha component geometry is designed entirely by the generator network ofthe GAN), the techniques associated with FIG. 9 combine thecomputational result from landmark-based automation tools (e.g., theautomation tools described in WO 2020/240351, filed May 20, 2020) or amanually generated geometry with neural network-based fine-tuning tofinalize the design with any last-mile tweaks (to form the updatedcomponent model) that may be beneficial to the dental restorationtreatment of patient 102. The GAN of FIG. 9 provides fast convergencetimes, enabling computing device 150 to provide the benefits of both theinitial geometry design using landmark-based techniques and neuralnetwork-based geometry refinements in computationally fast manner togenerate the updated component model.

The GAN of FIG. 9 enables generator network training even in cases ofhaving limited numbers of previously generated models to use as examplesfor training. In this way, the GAN of FIG. 9 leverages engineeredfactors of the originally designed geometry in cases in which trainingdata are limited, while implementing the benefits of neuralnetwork-based design for last-mile tweaking of the original design. Incomparison to the GAN of FIG. 4 , the GAN of FIG. 9 provides thegenerator network an additional input (in both the training phase andthe execution phase) with an original appliance geometry that may needtweaking to bring it to final form (in the form of the updated model orupdated component 3D mesh) for fabrication (e.g., via 3D printing) bymanufacturing system 194.

FIGS. 10-12 are directed to aspects of this disclosure that describesystems configured to display proposed dental restorations to patient102 via an automated design process using generative modeling. Accordingto these aspects of this disclosure, neural network engine 184 utilizesdata collected from dental practitioner 106 and/or other trainedclinicians/technicians to train a neural network configured togeneratively model the proposed dental anatomy of patient 102. Forinstance, neural network engine 184 may train the neural network tolearn attributes of an acceptable dental restoration in a data-drivenmanner. Examples of this disclosure are described with respect togenerating a unique two-dimensional (2D) image of a post-restorationproposed dental anatomy for a single patient (patient 102 in theseexamples).

The neural network-based generative display techniques of thisdisclosure are described below with respect to dental restoration, byway of the non-limiting example of 2D image generation for a(post-restorative) proposed dental anatomy. However, it will beunderstood that the neural network-based generative display techniquesof this disclosure can also be applied with respect to other areas, suchas to assist in 3D printing of ceramic and/or composite dental crowns,etc. The goal of various dental restorative treatments discussed hereinis to provide patient 102 with a composite restoration for damaged orunaesthetic teeth at a low cost with minimal invasiveness, or to providedental restoration for other sub-optimal conditions associated with thecurrent dental anatomy of patient 102.

Patient 102 (or any other patient) interested in dental restoration mayhave his/her current dental anatomy scanned at clinic 104. The neuralnetwork-based generative modeling techniques of this disclosure providea process-fast and data-precise 2D image view of the post-restorationproposed dental anatomy, custom-tailored for a given patient (patient102 in this particular example). The neural network-based generativemodeling techniques of this disclosure mitigate significant lead timeand costs from existing solutions for dental restoration planning

To improve data precision with respect to generative modeling ofpost-restoration 2D imaging of the proposed dental anatomy of patient102, neural network engine 184 may incorporate dental restoration styles(e.g., youth, geriatric, natural, oval, etc.) into the training data ifstyle information is available for past cases. In these and/or otherexamples, neural network engine 184 may incorporate one or more ofaccepted “golden proportion” guidelines for the size of teeth, accepted“ideal” tooth shapes, patient preferences, practitioner preferences,etc. into the training data used to train the neural network. Ifdifferent styles are available in the training dataset, then patient 102may have the ability to view different restoration options generated bythe algorithm in different styles. In other words, neural network engine184 may generate different style options with respect to the proposedpost-restorative dental anatomy of patient 102 based on differentlystyled outcomes in past cases.

By training the neural network with these dental restoration-relatedtraining data (usually over numerous training iterations forfine-tuning), neural network engine 184 improves data precision withrespect to generatively modeling the proposed dental anatomy of patient102, reduces the computational resource expenditure at runtime (byexecuting a precisely trained neural network), and reduces the overallprocess time for generating the dental restoration treatment plan, byreducing the iterations needed to correct or fine-tune multiple attemptsat planning a treatment for a given patient for a single round of dentalrestorative treatment.

Computing devices 150 and 190 provide various user experience-relatedimprovements as well, by way of computing device 150 implementing theneural network-based generative modeling techniques of this disclosure.For example, dental practitioner 106 may present patient 102 with 2Dimage of the proposed post-restoration dental anatomy by generating the2D image relatively quickly (and potentially during the same patientencounter) upon scanning the current dental anatomy of patient 102. Insome examples, dental practitioner 106 may synthesize differentpost-restorative outcomes (e.g., using different styles or otherpreference-related factors) to enable patient 102 to view differentoptions to aid in choosing a dental restoration plan.

In some examples, dental practitioner 106 may provide a “pre-approved”target for the generation of a 3D restoration file, which can be used inthe design and/or manufacture processes of dental appliance 112.Providing pre-approved plan information (which neural network engine 184may obtain from practitioner preferences library 168 or other sources)enables neural network engine 184 to train the neural network togenerate custom-tailored dental restoration models with reduced amountsof practitioner input, compressing the production process for customproducts.

Because patient 102 can visualize a potential post-restoration outcomeof his/her own dental anatomy rather than past cases of other patients,neural network engine 184 leverages training data to provideindividualization as a user experience improvement in these examples, aswell. The generative modeling techniques of this disclosure may beapplicable to areas other than dental restoration in which a patient isinterested in a unique or customized solution as well, such as withrespect to respirators, bandaging, etc.

According to some examples of this disclosure, neural network engine 184uses a GAN to generate the 2D image of the proposed dental anatomy ofpatient 102 for the post-restorative treatment stage. As describedabove, GANs utilize a pairing of differentiable functions, often deepneural networks, with the goal of learning the generation of an unknowndata distribution. The first network, known as the generator network,produces a data sample given some input (e.g., random noise, conditionalclass label, etc.). The second network, known as the discriminatornetwork, attempts to classify the data generated by the generator from areal data point coming from the true data distribution.

As part of the training, the generator network continuously tries tospoof (or “fool” or “trick”) the discriminator into classifying datagenerated de novo as “real.” As the success rate of spoofing thediscriminator network using the generated data becomes more frequent,the training outputs of the generator network become more realistic. Insome examples of the generative 2D modeling techniques of thisdisclosure, neural network engine 184 uses a conditional GAN (cGAN),where the generator network is conditioned on a 2D rendered image of a2D scan of the current dental anatomy of patient 102.

The generator network, which is a CNN in some non-limiting examples,takes as input a rendered 2D image of the current (pre-restoration)dental anatomy of patient 102, and generates a 2D image of how theproposed (post-restoration) dental anatomy of patient 102 would appear,based on the current state of the adversarial training of the generatornetwork. The generator network may also accept as input data (dependingon availability and/or relevance) additional information such as whichteeth are to be restored, the restoration style (e.g. youth, geriatricnatural, oval, etc.), etc.

The discriminator network, which may also be a CNN in some examples,receives a pair of 2D images as input. The first pair contains arendered 2D image of the pre-restoration dental anatomy of a patient anda rendered 2D image of the true restoration performed by a clinician(which is classified as a “real” or “ground truth” pairing) for the samepatient. The second pair of images contains a rendered 2D image ofbefore restoration and a generator network-generated restoration. Thegenerator and discriminator networks are trained simultaneously, in analternating fashion, improving upon each other to reach a shared goal ofa precisely trained generator network.

In some embodiments, neural network engine 184 implements the generatornetwork of the 2D restoration image aspects of this disclosure as anencoder-decoder CNN. In these examples, the generator network reducesthe dimensionality of the input image, and then expands thedimensionality back up to the original dimensionality (e.g, via asequence of downsampling and upsampling, or in other ways). Thegenerator network in these examples may also be referred to as a“U-Net.” As used herein, a “U-Net” refers to a type of encoder-decoderarchitecture in which the feature maps from the encoder are concatenatedonto the respective feature maps in the decoder.

In a traditional GAN, the discriminator network receives either a realimage (coming from the input dataset of images) or a syhtnesized image(which was produced by the generator). The output of the discriminatornetwork is a probability in the range [0,1] representing the probabilityof the input image being a real image (coming from the dataset). In someimplementations of the 2D restoration image aspects of this disclosure,the discriminator network is a “patchGAN” discriminator network.

While a typical discriminator network outputs a single valuerepresenting the perceived realness of the input, a patchGANdiscriminator network outputs an [n×n] matrix, where each elementrepresents the perceived realness of a corresponding patch of the input.The perceived realness as represented by each element of the [n×n]output matrix represents the probability of the corresponding patch ofthe input image being part of a real or ground truth image. Thediscriminator network is implemented internally as a CNN.

FIG. 10 is a conceptual diagram illustrating the symbiotic trainingprocesses for the generator network and discriminator network of a cGANconfigured to render 2D images of a proposed dental anatomy for patient102 according to aspects of this disclosure. Aspects of FIG. 10 alsoillustrate the manner in which various data are handled by the generatorand discriminator networks. In FIG. 10 , “G” denotes the generatornetwork of the cGAN, and “D” denotes the discriminator network of thecGAN. The pre-restoration 2D image (which is the input to G and half ofthe image pairing supplied by G to D) is denoted by “x”. “G(x)” denotesthe 2D image of the proposed restoration generated by G, given x as thepre-restoration input. The 2D rendered image of the actually performeddental restoration (or “true” or “ground truth” image of thepost-restoration dental anatomy) is denoted by “y.”

The particular use case scenario illustrated in FIG. 10 is associatedwith an unsuccessful iteration during the multi-iteration trainingprocess of G. As shown in FIG. 10 , D outputs a decision that thecombination of G(x) with x is a “fake.” In contrast, and as intendedwith respect to adversarial cGAN training, D outputs a decision of“true” upon evaluating an input combination of x and y. In someexamples, if D is a sufficiently trained and refined network, then afterG is adversarially trained more precisely over future iterations of cGANtraining, G may generate instances of G(x) that, when fed into D with x,successfully spoof D into outputting “true” decisions.

In these examples, upon G reaching this level of training, neuralnetwork 184 may begin executing G to generate, from an input of x,proposed 2D images of post-restoration dental anatomies of patient 102.In some examples, because G and D are trained in tandem, both networksmay be untrained for similar periods of time. In these cases, both G andD may undergo training until G passes qualitative inspection, such as bypassing a visual inspection by dental practitioner 106 or anotherclinician. Following the formats used above, ‘x’ represents the 2Dpre-restoration image, ‘y’ is the ground truth 2D restored image, andG(x) is the image generated by the G given the pre-restoration imageinput. The total loss term used in some examples is a combination of L1loss and GAN loss, given by equation (1) below:

total_(error) _(G) =gan _(toss) _(g) +(λ_(L1) *L1_(toss))  (1)

The L1 loss is the absolute value of the difference between y and G(x),with the total loss being applied to G, but not to D. The calculation ofthe L1 loss is given by equation (2) below, where λ_(L1) is 10 in thisparticular example, although it will be appreciated that λ_(L1) can haveother values in other examples consistent with this disclosure.

L1_(toss)=λ_(L1) *abs(y−G(x))

By leveraging the communicative connection to computing device 150 overnetwork 114, computing device 190 may provide a chair-side applicationthat enables dental practitioner 106 to show patient 106 a 2D renderingof one or more proposed restoration plans, often during the same patientvisit during which the scan of the current dental anatomy is taken (andsometimes, very shortly or immediately after the scan is taken). Ratherthan displaying past cases of other patients or a general model whichcovers hypothetical patients, computing device 190 may use a cGANexecuted by neural network engine 184 to output a custom rendering ofthe proposed dental anatomy for one or more post-restoration scenariosapplicable specifically to the current dental anatomy and treatmentplan(s) for patient 102.

That is, neural network engine 184 may implement generative modelingtechniques of this disclosure in such a way that dental practitioner 106can leverage cloud computing interactions to render a 2D image ofproposed dental anatomies for one or more dental restoration plansapplicable specifically to the current dental anatomy of patient 102.From the perspective of those at clinic 104, given a scan of the currentdental anatomy of patient 102, computing device 190 quickly (or almostimmediately) processes the scan and leverages cloud computingcapabilities to render 2D images of one or more post-treatment dentalanatomy images specific to the dentition of and treatment optionsavailable to patient 102.

In this way, neural network engine 184 may implement the generativemodeling techniques of this disclosure entirely in the image domain,without requiring the potentially time-consuming generation of a 3Dmesh. Upon approval (e.g., by patient 102 and/or by dental practitioner106), computing device 190 may communicate the generated 2D image vianetwork 114 to computing device 192, enabling manufacturing system 194to generate a 3D mesh of dental appliance 112, or to directlymanufacture dental appliance 112.

In some examples, neural network engine 184 may generate or regeneratethe 2D image of the proposed dental restoration to incorporatepatient-specified modifications, such as a restoration style selectionor other parameters. In one such example, neural network engine 184 mayimplement a feedback loop within the cGAN to accommodatepatient-provided or practitioner-provided modifications with respect torestoration style, tooth shaping, etc.

In one example, the trained generator network of the cGAN may enable atechnician to create a 3D mesh from the 2D image output by the trainedgenerator network. In another example, the 3D mesh can be automaticallygenerated from the 2D image of the proposed dental anatomy output by thetrained generator network. In one or more of these examples, the 3D meshmay be used as an input to the systems described above with respect toFIGS. 3 & 4 . In some examples, dental practitioner 106 or otherclinician at clinic 104 may use image capture hardware (e.g., a stillcamera or video camera) to obtain a photograph of the current dentalanatomy of patient 102. In these examples, computing device 190 maygenerate a rendering of the current dental anatomy of patient 102 usingthe captured photograph.

As such, according to various examples of this disclosure, computingdevices 190 and 150 may obtain a 2D image (whether a dental scan or aphotograph) of a 3D object (in this case, the dentition of patient 102),and use the 2D image to generate another 2D image of the proposed dentalanatomy (or a portion thereof) with respect to the proposed dentalrestoration treatment of patient 102. In this way, computing devices 190and 150 may enable dental restoration modeling in a computationallylight and process-fast manner, while maintaining data precision withrespect to the dental restoration modeling, using the neural networktraining mechanisms of this disclosure.

In some examples, neural network engine 184 may execute the trainedversion of generator network G as an input generation system withrespect to the neural networks illustrated in FIGS. 3 & 4 . Forinstance, neural network engine 184 may augment the transform matrixinputs to the neural network of FIG. 3 and/or the tooth mesh inputs tothe generator network of FIG. 4 with a 2D image of the proposed dentalanatomy of patient 102. In these instances, neural network engine 184avails of the output of the trained version of generator network G totrain the neural network of FIG. 3 and/or the generator Graph CNN ofFIG. 4 in a more holistic way, accounting for effects on a greaterproportion of the archform of the dentition of patient 102 for whichplacement and/or geometry information are being generated.

FIG. 11A illustrates the input and output of a cGAN-trained generatornetwork configured to generate a 2D image of a proposed dental anatomyusing a 2D rendering of a current dental anatomy of patient 102. Currentdental anatomy image 1102 illustrates the 2D rendering of the currentdental anatomy of patient 102. Upon being cGAN trained (e.g., bysuccessfully fooling the discriminator network at least a thresholdnumber of times), the generator network (‘G’) of FIG. 10 uses currentdental anatomy image 1102 to generate proposed dental anatomy image1104.

Current dental anatomy image 1102 is a 2D rendering of thepre-restoration dentition of patient 102. Proposed dental anatomy image1104 is a 2D rendering of a projection of the end result of one proposeddental restoration treatment plan for patient 102. As such, FIG. 11shows a working example of one use case scenario in which a cGAN-trainediteration of generator network G performing the generative modelingtechniques of this disclosure.

FIG. 11B illustrates a comparison between current dental anatomy image1102, proposed dental anatomy image 1104, and a ground truth restorationimage 1106.

FIG. 12 illustrates menus that computing device 190 may display as partof a graphical user interface (GUI) that includes current dental anatomyimage 1102 and/or proposed dental anatomy image 1104. Data menu 1202presents dental practitioner 106 or another clinician with variousoptions to manipulate the content of the generative modeling. In theexample of FIG. 12 , data menu 1202 presents options of selectable testcases upon which to build the dental restoration plan. Data menu 1202also includes tooth options, which enable dental practitioner 106 toselect particular teeth from current dental anatomy image 1102 for whichreconstruction is to be modeled.

Visual option menu 1204 enables dental practitioner 106 to adjustvarious viewing parameters with respect to the display of current dentalanatomy image 1102 and/or proposed dental anatomy image 1104. Dentalpractitioner 106 may adjust various viewing parameters via visual optionmenu 1204 to enable patient 102 to better see the details of proposeddental anatomy image 1204.

In this way, dental practitioner 106 or other clinicians may operatecomputing device 190 at clinic 104 to effectuate cloud interactions overnetwork 114, thereby leveraging the neural network-based generativemodeling functionalities provided by computing device 150. By operatingdata menu 1202, dental practitioner 106 may provide the restorationmodeling parameters that neural network engine 184 uses in generatingproposed dental anatomy image 1204. By operating visual option menu1204, dental practitioner 106 uses computing device 190 as an onsitedisplay that customizes the viewing parameters of proposed dentalanatomy image 1204 to suit the viewing needs and preferences of patient102.

FIGS. 13A & 13B are conceptual diagrams illustrating example moldparting surfaces, in accordance with various aspects of this disclosure.Neural network engine 184 may generate mold parting surface 1302 basedon landmarks such as a midpoint of each tooth and points betweenadjacent teeth (e.g., points of contact between adjacent teeth and/orpoints of closest approach between adjacent teeth) for each slice. Insome examples, neural network engine may generate a 3D mesh of moldparting surface 1302 as part of the neural network-based placementgeneration techniques of this disclosure. Additional details of how moldparting surface 1302 can be used with respect to appliance manufactureare described in WO 2020/240351, filed May 20, 2020.

FIG. 14 is a conceptual diagram illustrating an example gingival trimsurface, in accordance with various aspects of this disclosure. Gingivaltrim surface 1402 may include a 3D mesh that trims an encompassing shellbetween the gingiva and the teeth in the illustrated dental anatomy.

FIG. 15 is a conceptual diagram illustrating an example facial ribbon,in accordance with various aspects of this disclosure. Facial ribbon1502 is a stiffening rib of nominal thickness that is offset faciallyfrom the shell. In some instances, the facial ribbon follows both thearchform and the gingival margin. In one instance, the bottom of thefacial ribbon falls no farther gingivally than the gingival trimsurface.

FIG. 16 is a conceptual diagram illustrating an example lingual shelf1602, in accordance with various aspects of this disclosure. Lingualshelf 1602 is a stiffening rib of nominal thickness on the lingual sideof the mold appliance, inset lingually and following the archform.

FIG. 17 is a conceptual diagram illustrating example doors and windows,in accordance with various aspects of this disclosure. Windows1704A-1704H (collectively, windows 1704) includes an aperture thatprovides access to the tooth surface so that dental composite can beplaced on the tooth. A door includes a structure that covers the window.The shape of the window may be defined as a nominal inset from theperimeter of the tooth when viewing the tooth facially. In someinstances, the shape of the door corresponds to the shape of a window.The door may be inset to create clearance between the door and window.

FIG. 18 is a conceptual diagram illustrating example rear snap clamps,in accordance with various aspects of this disclosure. Neural network184 may determine one or more characteristics (e.g., placement-relatedor geometry-related characteristics) of rear snap clamps 1802A & 1802B(collectively, “rear snap clamps 1802”). Rear snap clamps 1802 may beconfigured to couple a facial portion of dental appliance 112 with alingual portion of dental appliance 112. Example characteristics includeone or more of a size, shape, position, or orientation of rear snapclamps 1802. Position information for rear snap clamps 1802 may be alongthe archform on opposite ends of the archform (e.g., a first snap clampat one end and a second snap clamp at another end). In some examples, afemale portion of rear snap clamps 1802 may be positioned on the lingualside of the parting surface and a male portion of rear snap clamps 1802may be positioned on the facial side.

FIG. 19 is a conceptual diagram illustrating example door hinges, inaccordance with various aspects of this disclosure. Neural networkengine 184 may determine one or more characteristics of door hinges1902A-1902F (collectively, door hinges 1902) as part of generatingplacement and/or geometry information for dental appliance 112 inaccordance with various aspects of this disclosure. Door hinges 1902 maybe configured to pivotably couple a door to dental appliance 112.Example characteristics include one or more of size, shape, position, ororientation of the respective door hinge(s) 1902. The neural networkexecuted by neural network engine 184 may position door hinges 1902based on a position of another pre-defined appliance feature, in somenon-limiting use case scenarios. For example, the neural network mayposition each door hinge 1902 at the midline of a corresponding door. Inone use case scenario, the female portion of a respective door hinge1902 may be positioned to anchor to the facial portion of dentalappliance 112 (e.g., towards the incisal edge of a respective tooth) andthe male portion of the same door hinge 1902 may be positioned to anchorto the outer face of the door.

FIGS. 20A & 20B are conceptual diagrams illustrating example door snaps,in accordance with various aspects of this disclosure. Neural networkengine 184 may determine one or more characteristics of door snaps2002A-2002F (collectively, “door snaps 2002”), such as placementcharacteristics and/or geometry characteristics. Example characteristicsinclude one or more of size, shape, position, or orientation of the doorsnaps 2002. In some examples, the neural network executed by neuralnetwork engine 184 may determine the position of door snaps 2002 basedon a position of another pre-defined appliance feature. In one example,the neural network may generate a placement profile that positions eachdoor snap 2002 at the midline of a corresponding door. In one example,the position of the female portion of a particular door snap 2002 may beanchored to an outer face of the door and extends downward toward thegingiva. In another example, the male portion of a particular door snap2002 may be anchored to the gingival side of the facial ribbon.

FIG. 21 is a conceptual diagram illustrating an example incisal ridge,in accordance with various aspects of this disclosure. Incisal ridge2102 provides reinforcement at the incisal edge.

FIG. 22 is a conceptual diagram illustrating an example center clip, inaccordance with various aspects of this disclosure. Center clip 2202aligns the facial portion and the lingual portion of the dentalappliance with one another.

FIG. 23 is a conceptual diagram illustrating example door vents, inaccordance with various aspects of this disclosure. Door vents2302A-2302B (collectively, “door vents 2302”) transport excess dentalcomposite out of the dental appliance.

FIG. 24 is a conceptual diagram illustrating example doors, inaccordance with various aspects of this disclosure. In the example ofFIG. 24 , a dental appliance includes door 2402, door hinge 2404, anddoor snap 2406.

FIG. 25 is a conceptual diagram illustrating an example diastema matrix,in accordance with various aspects of this disclosure. Diastema matrix2502 includes handle 2504, body 2506, and wrapping portion 2508.Wrapping portion 2508 is configured to fit in the interproximal regionbetween two adjacent teeth.

FIG. 26 is a conceptual diagram illustrating an example manufacturingcase frame and an example dental appliance, in accordance with variousaspects of this disclosure. Manufacturing case frame 2602 is configuredto support one or more parts of a dental appliance. For example, themanufacturing case frame 2602 may detachably couple a lingual portion2604 of a dental appliance, a facial portion 2606 of the dentalappliance, and a diastema matrix 2608 to one another via case framesparring 2610. In the example of FIG. 26 , case frame sparring 2610 tiesor couples the parts 2604, 2606, and 2608 of the dental appliance to themanufacturing case frame 2602.

FIG. 27 is a conceptual diagram illustrating an example dental applianceincluding custom labels, in accordance with various aspects of thisdisclosure. Custom labels 2702-2708 may be printed on various parts ofthe dental appliance and includes data (e.g., a serial number, a partnumber, etc.) identifying a respective part of the dental appliance.

Various examples have been described. These and other examples arewithin the scope of the following claims.

1-43. (canceled)
 44. A computing device comprising: an input interfaceconfigured to receive one or more three-dimensional (3D) tooth meshesassociated with a current dental anatomy of a dental restoration patientand a 3D component mesh representing a generated geometry for a dentalappliance component; and a neural network engine configured to: providethe one or more 3D tooth meshes and the 3D component mesh received bythe input interface as inputs to a neural network trained with trainingdata comprising ground truth dental appliance component geometries and3D tooth meshes of corresponding dental restoration cases; and executethe neural network using the provided inputs to produce an updated modelof the dental appliance component with respect to the current dentalanatomy of the dental restoration patient.
 45. The computing device ofclaim 44, wherein the neural network is a generative adversarial network(GAN) comprising a generator network and a discriminator network, andwherein to execute the neural network to produce the updated model ofthe dental appliance component with respect to the current dentalanatomy of the dental restoration patient, the neural network engine isconfigured to provide the one or more 3D tooth meshes and the 3Dcomponent mesh received by the input interface as input to the generatornetwork of the GAN.
 46. The computing device of claim 45, wherein theneural network engine is configured to train the GAN by: executing thegenerator network of the GAN to generate an updated geometry of thedental appliance component using the one or more 3D tooth meshes and the3D component mesh; providing the updated geometry of the dentalappliance component, the one or more 3D tooth meshes, and the 3Dcomponent mesh as inputs to the discriminator network of the GAN; andexecuting the discriminator network to output a probability of theupdated geometry representing a ground truth geometry.
 47. The computingdevice of claim 44, wherein the neural network is further trained withplacement information of the ground truth dental appliance componentgeometries as part of the training data.
 48. The computing device ofclaim 44, wherein the dental appliance component comprises one or moreof: a mold parting surface, a gingival trim, a door, a window, a facialribbon, an incisal ridge, a lingual shelf, a diastema matrix, a caseframe, a part label; a door hinge, a door snap, a door vent, a snapclamp, or a center clip.
 49. A method comprising: receiving, at an inputinterface, one or more three-dimensional (3D) tooth meshes associatedwith a current dental anatomy of a dental restoration patient and a 3Dcomponent mesh representing a generated geometry for a dental appliancecomponent; providing, by a neural network engine communicatively coupledto the input interface, the one or more 3D tooth meshes and the 3Dcomponent mesh received by the input interface as inputs to a neuralnetwork trained with training data comprising ground truth dentalappliance component geometries and 3D tooth meshes of correspondingdental restoration cases; and executing, by the neural network engine,the neural network using the provided inputs to produce an updated modelof the dental appliance component with respect to the current dentalanatomy of the dental restoration patient.
 50. The method of claim 49,wherein the neural network is a generative adversarial network (GAN)comprising a generator network and a discriminator network, and whereinexecuting the neural network to produce the updated model of the dentalappliance component with respect to the current dental anatomy of thedental restoration patient comprises providing, by the neural networkengine, the one or more 3D tooth meshes and the 3D component meshreceived by the input interface as input to the generator network of theGAN.
 51. The method of claim 50, further comprising training, by theneural network engine the GAN by: executing the generator network of theGAN to generate an updated geometry of the dental appliance componentusing the one or more 3D tooth meshes and the 3D component mesh;providing the updated geometry of the dental appliance component, theone or more 3D tooth meshes, and the 3D component mesh as inputs to thediscriminator network of the GAN; and executing the discriminatornetwork to output a probability of the updated geometry representing aground truth geometry.
 52. The method of claim 49, wherein the neuralnetwork is further trained with placement information of the groundtruth dental appliance component geometries as part of the trainingdata.
 53. The method of claim 49, wherein the dental appliance componentcomprises one or more of: a mold parting surface, a gingival trim, adoor, a window, a facial ribbon, an incisal ridge, a lingual shelf, adiastema matrix, a case frame, a part label; a door hinge, a door snap,a door vent, a snap clamp, or a center clip.
 54. (canceled)
 55. Anon-transitory computer-readable storage medium encoded withinstructions that, when executed, cause one or more processors of acomputing system to: receive one or more three-dimensional (3D) toothmeshes associated with a current dental anatomy of a dental restorationpatient and a 3D component mesh representing a generated geometry for adental appliance component; provide the one or more 3D tooth meshes andthe 3D component mesh received by the input interface as inputs to aneural network trained with training data comprising ground truth dentalappliance component geometries and 3D tooth meshes of correspondingdental restoration cases; and execute the neural network using theprovided inputs to produce an updated model of the dental appliancecomponent with respect to the current dental anatomy of the dentalrestoration patient.